[ci] Extend the hardware-test skill with the flows learned since it was added (#21011)

This commit is contained in:
J. Nick Koston
2026-10-18 15:09:01 -10:00
committed by GitHub
parent 490d4723ca
commit 73ba726838
52 changed files with 2703 additions and 129 deletions
+33 -10
View File
@@ -15,19 +15,26 @@ driven, what the logs show, and the same flow on a `dev` build when a quirk coul
1. **Serial ports cannot be shared.** One bounded capture per port, stopped before any flash on that
port, never an open-ended `esphome logs`. Stop a capture with `pkill -f "[s]erial_capture.py <port> "`,
naming the port so a capture on another board survives; the bracket keeps the pattern from matching
the shell that runs it. Never run `esphome logs` in a
`while true` loop: the loop respawns it and holds the port.
the shell that runs it. When more than one agent session shares the bench, run a copy of the script
under a session-specific name (`cap<topic>.py`) so another session's `pkill` cannot take your capture
down. Never run `esphome logs` in a `while true` loop: the loop respawns it and holds the port.
2. **Credentials stay local.** Put a `secrets.yaml` next to the scratch configs and reference it with
`!secret`. Never paste a config with credentials into a PR, never commit scratch configs or logs.
`!secret`. A copy of a device's own config keeps using the secrets file next to that config: two
checkouts can hold different networks, and the wrong one leaves the device unreachable after the OTA. Never paste a config with credentials into a PR, never commit scratch configs or logs.
3. **Build from the tree under test.** `python -m esphome` puts the current directory first on
`sys.path`, so run it with the checkout or worktree as the working directory and pass the yaml by
its absolute path. A venv `esphome` entry point may import an installed snapshot instead of the
checkout. Confirm with `python -c 'import esphome; print(esphome.__file__)'`, and for an A/B grep
the build's `src/` for the change before trusting any number.
the build's `src/` for the change before trusting any number. Tool calls sent together share one
shell, so a `cd` in one can land another's compile in the wrong tree: pass `PYTHONPATH=<worktree>` on
every compile and upload command and print `esphome.__file__` into the compile log.
4. **PR build and baseline build get different `name:` values.** Builds share
`.esphome/build/<name>`, so the same name means `esphome upload` can flash the wrong binary. Saved
Wi-Fi credentials are keyed by the name too, so a test that must start unprovisioned gets a fresh
name every run.
Wi-Fi credentials are keyed by the config hash when the config has `networks:`, so a test that must
start unprovisioned gets a fresh name every run. A config with `ap:` only and no `networks:` uses one
fixed key shared by every such config on the board, so a new name does not help there: wipe once at
boot with an `on_boot` lambda that resets the preferences and saves a marker key
(`templates/dut-ap-only-wipe-once-bk72xx.yaml`).
5. **Flash and RAM deltas are measured against the PR's merge base** (`git merge-base origin/dev HEAD`
checked out in a throwaway worktree), with the baseline commit named in the table. When a delta
looks odd, diff symbol sizes (`nm -S -C --size-sort` from the matching toolchain) before quoting it.
@@ -73,7 +80,7 @@ driven, what the logs show, and the same flow on a `dev` build when a quirk coul
| ESP32-S3, C3, C6, H2 native USB | `esphome upload --device /dev/cu.usbmodem*` | `logger: hardware_uart: USB_SERIAL_JTAG`. The USB console can wedge when the app takes over the console (0 bytes, esptool `No serial data received`); replug, or watch through `esphome logs --device <name>.local` |
| Pico W / Pico 2 W | `esphome upload --device <port>` (1200 baud touch); first flash ever via BOOTSEL and `--device /Volumes/RPI-RP2` or `RP2350` | port re-enumerates after upload: `ports.py --wait 2e8a` then capture |
| BW15 kit (RTL8720CF, rtl87xx) | `~/.platformio/penv/.libretiny/bin/python $SKILL/scripts/flash_rtl87xx_kit.py flash write -d <port> <build>/.pioenvs/<name>/*.uf2`. Plain `esphome upload` and plain ltchiptool time out because opening the port asserts DTR/RTS and holds the chip in reset | `serial_capture.py --released`, never `esphome logs` |
| BK7238 / CB3S (bk72xx, RTS on CEN) | `~/.platformio/penv/.libretiny/bin/python $SKILL/scripts/flash_bk72xx.py flash write -d <port> <uf2>` (about 75 s) | logs are on UART2 by default; `logger: hardware_uart: UART1` puts them on the flashing adapter. Opening the port resets the chip, so keep one capture open for the whole test |
| BK7238 / CB3S (bk72xx, RTS on CEN) | `~/.platformio/penv/.libretiny/bin/python $SKILL/scripts/flash_bk72xx.py flash write -d <port> <uf2>` (about 75 s) | logs are on UART2 by default; `logger: hardware_uart: UART1` puts them on the flashing adapter. `serial_capture.py --reset` gives a chip-enable reset; a plain open may or may not reset it, check for the bootloader banner |
| LN882H | ltchiptool forces 1 Mbaud for writes; weak adapters need `LN882H_YM_BAUDRATE` patched to 115200 in the penv's `ln882htool.py`; short BOOT to GND while the tool connects | OTA after the first flash |
| nRF52840 Dongle (PCA10059) | Nordic `nrfutil nrf5sdk-tools dfu usb-serial` on the bootloader port, or `dfu: true` in the config so the 1200 baud touch pin-resets into the bootloader; adafruit-nrfutil is refused | app has two CDC ports, ESPHome log on the second |
| Adafruit-style nRF52 (UF2 bootloader) | `UF2BOOT` volume or adafruit-nrfutil; `dfu: true` works through GPREGRET | single CDC port |
@@ -91,7 +98,8 @@ ESP-IDF OTA image: `.esphome/build/<name>/build/firmware.ota.bin`. LibreTiny OTA
deadline. OUT is truncated unless `--append` is given, so counts over it cover one run.
- `flash_rtl87xx_kit.py`: ltchiptool wrapper for the BW15 kit whose CH340 drives CEN (RTS) and the boot
strap (DTR). `flash read` needs the family `realtek-ambz2`; `flash write` does not.
- `flash_bk72xx.py`: ltchiptool wrapper for BK72xx boards whose RTS drives CEN.
- `flash_bk72xx.py`: ltchiptool wrapper for BK72xx boards whose RTS drives CEN. It misses the link now
and then; retry up to three times.
- `wifisave_hammer.sh KIND DUT_PORT DUT_YAML DUT_TREE CLIENT_PORT CLIENT_YAML CLIENT_TREE [SETTLE]`:
the fallback AP plus captive portal hammer end to end (compile, flash, capture, counts) with the
templates below. KIND is `esp`, `pico`, `bw15` or `bk`. It ends with `RESULT PASS` (exit 0),
@@ -100,6 +108,13 @@ ESP-IDF OTA image: `.esphome/build/<name>/build/firmware.ota.bin`. LibreTiny OTA
(exit 2: a precondition was missed, the stimulus was insufficient, or a capture did not last the run)
or exits 1 when a build, flash or port step failed.
Per-topic scripts, each described in the matching `reference/recipes/<component>.md`: `online_image/`
(server, image generator, API driver), `zwave_proxy/` (checksum verifier, device info), `usb_cdc_acm/`
(host driver, Zephyr driver probe), `rtsp_server/` (protocol client, raw rate reader, measurement set,
API image grab), `cse7766/` (API observer with ping), `zephyr_coredump/` (reader flow, decode with the
Zephyr tools), `libretiny_prefs/` (rtc preference harness header), `libretiny_crash/` (BK72xx crash probe header and
capture decoder).
## Templates in `$SKILL/templates`
- `dut-fallback-ap.yaml` (package): a device that never finds its network, falls back to its AP and
@@ -107,17 +122,25 @@ ESP-IDF OTA image: `.esphome/build/<name>/build/firmware.ota.bin`. LibreTiny OTA
- `client-wifisave.yaml` (package): joins that AP, posts N bogus `/wifisave` requests, then the real
credentials from `!secret`. `example-client.yaml` is the per-run file. `bogus_saves: "1"` gives a
plain provisioning check.
- Per-topic device configs, each described in `reference/recipes/<component>.md`: `online-image-dut.yaml`,
`usb-cdc-acm-dongle-dut.yaml`, `nrf52-preferences-dut.yaml`, `rtsp-waveshare-s3-eth-dut.yaml`,
`waveshare-s3-eth-camdiag.yaml`, `s31-cse7766-dut.yaml`, `nrf52-coredump-crash-dut.yaml`,
`bk72xx-rtc-prefs-dut.yaml`, `bk72xx-crash-probe-dut.yaml`, `dut-ap-only-wipe-once-bk72xx.yaml`.
## When something looks wrong
- A capture stopped early or a line is missing: check the timestamps before trusting it.
- The client never joins the AP: another board may still run an AP with the same SSID, or the device
booted with saved credentials because the name was reused. One AP device at a time on the bench.
- Builds come out byte-identical between branches: the wrong tree was imported (rule 3).
- Builds come out byte-identical between branches: the wrong tree was imported (rule 3). The reverse
too: two binaries that differ only in the build timestamp and in `__LINE__` values of log calls below
the changed code are the same build; map the offsets with `objdump -h` and `addr2line` first.
- A boot the host cannot capture (a power cut, an OTA-only board): keep the boot report in a flash
preference and repeat it in a periodic alive line, as `scripts/libretiny_prefs/ltprefs.hpp` does.
- Multicast from the host never arrives on some access points: test it device to device. Broadcast
is lossy under Wi-Fi power save, unicast is not. Throttle UDP floods to a fixed packet rate.
- `api:` reboots a board after 15 min without a client: `reboot_timeout: 0s` on `api:` and `mqtt:` for
soak tests.
`reference/boards.md` has per-chip facts and crash decoding; `reference/recipes.md` has the proven
`reference/boards.md` has per-chip facts and crash decoding; `reference/recipes/` has the proven
flows. Add new findings there.
@@ -88,8 +88,28 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
pulses the reset right before the link checks. A write takes about 75 s. Only a CEN or power reset
reaches the downloader; no software reboot does.
- ESPHome logs on UART2 by default; `logger: hardware_uart: UART1` puts them on the flashing adapter.
- Opening the port resets the chip (bootloader banner on every open, even with DTR/RTS low), so keep one
capture open for the whole test and never reopen mid-phase.
- Opening the port may reset the chip (bootloader banner on open) or may not, depending on the adapter and
what pyserial does with DTR and RTS; `serial_capture.py --reset` pulses RTS on CEN and always does, and
`lt_get_reboot_reason()` reports that as Power-On. Check for the banner at the start of a capture before
assuming either.
- `flash_bk72xx.py` misses the link now and then (`Timed out attempting to link with chip`); retry up to 3
times before suspecting the board.
- Crashes: `__builtin_trap()` and a call into unmapped memory both end as an undefined-instruction trap
(`bk_misc_get_start_type()` 6, `lt_get_reboot_reason()` Crash); reads, unaligned loads and stores to
unmapped addresses never fault on the ARM968. Without Wi-Fi the SDK trap handler spins with interrupts off
for 10 s and the hardware watchdog resets the chip. With Wi-Fi connected a dev build hangs forever after a
crash, and never resets from a plain `portDISABLE_INTERRUPTS(); while (1) {}` either: the boot-time
watchdog is dead once Wi-Fi is up, for any lockup. The crash handler fix (#21012) re-arms the watchdog
from its wrapper, after which the reboot comes within a second (BK7238 and BK7231N).
- The SDK prints nothing on UART1 under `sdk_silent: all`; the bootloader then prints the banked
`cpsr/spsr/r13/r14` of SVC, IRQ, FIQ and SYS plus `ST:` (the retention register) before `go os_addr`.
A CEN or power reset wipes ITCM (reads `0xaaaaaaaa`), so only a watchdog reset keeps a record there. What
survives that reboot on the BK7238: ITCM (`0x3F0000..0x3F4000`) and the top 96 KB of SRAM
(`0x428000..0x440000`, the heap); the bootloader zeroes SRAM up to about `0x420000` and rewrites
`0x400024..0x4000FF` with its own register dump every boot, so neither a `.noinit` after `.bss` nor the
SDK's saved register banks are usable. Code is compiled Thumb, the SDK assembly is ARM; in an exception
handler only r0 to r3 are shared by all modes. BK7231N ITCM is `0x3FEE00..0x400000` (4.5 KB); a `.noinit`
placed there after `.itcm.code` works for a crash record.
- Callbacks run on the `async_tcp` task and log lines can lose a byte. Wi-Fi connect takes tens of ms;
radio enable 160 to 278 ms, disable about 200 ms.
@@ -123,6 +143,38 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
- Makerdiary MDK USB dongle: UF2 bootloader without SoftDevice, app at 0x1000, `UF2BOOT` volume,
adafruit-nrfutil, GPREGRET 0x57 works. RGB LED active low (red P0.23, green P0.22, blue P0.24),
button P0.18.
- Adafruit ItsyBitsy nRF52840 Express (UF2 bootloader with S140, bootloader port `239a:0051` plus an
`ITSY840BOOT` volume): the board DTS already has `cdc_acm_uart0`, so the logger port is the first CDC
port and a `usb_cdc_acm` interface 1 the second. A `dfu: true` build enters the bootloader on the
1200 bps touch (an app without it needs a double tap on RESET); `adafruit-nrfutil` over the bootloader
serial port can fail, copying `.pioenvs/<name>/zephyr/zephyr.uf2` to the volume works (the copy reports
an I/O error as the board reboots, which is normal). Reboot a running `dfu: true` app with a 2001 bps touch.
- The PCA10059 board DTS already defines a `cdc_acm_uart` node, so every ESPHome CDC port is one USB port
later than its number (log on `cdc_acm_uart0` is the second port). A third CDC instance needs
`CONFIG_USB_MAX_NUM_TRANSFERS` above 4 or it never moves data (see `recipes/usb_cdc_acm.md`).
- USB CDC boot output: the Zephyr legacy stack's `cdc_acm` TX ring is 1 KB and `uart_poll_out` drops when it
is full, so anything printed in one burst beyond about 1 KB (a coredump, a long table) is lost even with
the port open, and the logger warns `logger took a long time`. Print one line per `loop()` instead.
- Zephyr settings survive a reflash: key a `restore_value` global by a `run` substitution when a test needs
a never-saved value.
## Waveshare ESP32-S3-ETH with the OV5640 camera module
- Native USB console (`303a:1001`), ESP32-S3R8, 16 MB flash, 8 MB octal PSRAM, W5500 on SPI (MOSI 11,
MISO 12, SCLK 13, CS 14, INT 10, RST 9), TF card on 4..7, WS2812 on 21.
- Camera header (vendor schematic): XCLK 3, SIOD 48, SIOC 47, VSYNC 1, HREF 2, PCLK 39, D0..D7 = 41, 45,
46, 42, 40, 38, 15, 18. PWDN is tied to GND and RESET is an RC to 3.3 V on the board, so no reset or
power-down GPIO exists.
- GPIO8 switches the camera 3.3 V rail through a P-channel MOSFET: LOW = camera powered, gate pulled up so
the camera is off by default. `power_down_pin: GPIO8` only pulls it low 10 ms before the probe, which is
not enough; use `switch: gpio GPIO8 restore_mode: ALWAYS_OFF setup_priority: 800` plus `on_boot:
priority: 700` with `delay(1200)` and `esp32_camera: setup_priority: 200`
(`templates/rtsp-waveshare-s3-eth-dut.yaml`). `esphome config` rejects a config that uses GPIO8 for both the switch
and `power_down_pin` (pin reuse), so it is one or the other.
- The sensor at SCCB 0x3C answers only with XCLK running, so the ESPHome i2c scan at boot (before XCLK)
reporting no devices on 48/47 is expected. If the SCCB ID register (0x3100 = 0x78) reads fine but every
0x30xx register reads 0x00 (`Detected camera not supported`, 0x106), reseat the ribbon before blaming
software; `templates/waveshare-s3-eth-camdiag.yaml` runs that probe with LEDC XCLK and raw SCCB reads.
## ESP Thread Border Router board v1.2 (S3 host + H2 RCP)
@@ -1,117 +0,0 @@
# Proven test flows
Each flow names the pull requests it was used on, so their result tables show what to post.
## Captive portal `/wifisave` hammer (#20802, #20784)
Reproduces a `/wifisave` arriving during the STA cooldown, and proves a fix on every board.
Device under test: `templates/example-dut.yaml` (bogus network, `ap:` with `ap_timeout: 15s`,
`captive_portal:`). Client: `templates/example-client.yaml` on an ESP32 or a D1 mini; it joins the
device's AP, posts 15 bogus saves 3 s apart, then the real credentials. With the portal up the cooldown
is 30 s, so nearly every save lands inside one. Driver: `scripts/wifisave_hammer.sh`. Settle 40 s on
ESP boards, 90 s on the BW15 (its first attempt on the real network fails, see boards.md).
Counts that matter: `Requested WiFi Settings Change` (saves received), `Exiting cooldown early`, crash
markers, boots (`Log initialized`), and the client's `bogus save status 200` and `real save status 200`.
A healthy run: every save answered 200, one cooldown exit per save (two on the BW15, whose retry enters
the adapter restart), no panic, and the real credentials connect the device within about 10 s.
Fault injection for AP start failures: wrap `this->ap_setup_ = this->wifi_start_ap_(this->ap_)` in
`setup_ap_config_()` with an `HWTEST_AP_FAIL_COUNT` build flag
(`esphome: platformio_options: build_flags: -DHWTEST_AP_FAIL_COUNT=2`), raise the portal's "Captive
portal started" from LOGV to LOGI, and after provisioning curl the device's STA address: connection
refused proves the portal's server is gone.
## `reboot_timeout` with a fallback AP (#9958)
Only one board runs the AP at a time, or stations join the wrong one. AP probe: bogus network,
`ap: {ssid, password, ap_timeout: 5s}`, `reboot_timeout: 60s`, `logger: level: VERBOSE` (ESP32 and
ESP8266 log `AP client connected MAC=` at verbose), `on_boot: logger.log: BOOT`. Station probes:
`reboot_timeout: 0s`, `interval: 150s` running `wifi.disable`, `delay: 20s`, `wifi.enable` (60 s cycles
are too short to cover the due moment). Pass pattern: "Can't connect; rebooting" exactly 60 s after boot
with no station joined, no reboot for minutes while one is, then a reboot within a second of the last
disconnect. The BW15 restarts its AP on every STA retry and cannot be tested this way.
## MQTT (#20433, #17935, #20762, #20767, #20768)
A local mosquitto with `allow_anonymous true`, `log_type all` and `log_dest file`: the broker log gives
each subscription's QoS, the device address (`New client connected from <ip> as <name>-<mac>`) and exact
subscribe counts. Kill and restart mosquitto to exercise `on_disconnect` (reason 0) and resubscription.
Keep `reboot_timeout: 0s` on `api:` and `mqtt:`. The ESP-IDF outbox swings free heap about 2 kB right
after bulk subscribe and unsubscribe; read the heap seconds later over several rounds before calling a
leak. Spawned subscriptions need about 2.5 s to reach the broker before the next publish.
`esphome logs --device MQTT` only uses the MQTT transport when the config has no `api:`. Driver pattern: a
paho script publishes and polls the serial capture file.
## UDP, multicast, sockets (#20556 to #20559, #20585, #20590)
A listener on port 9 needs no privileges on macOS. Test multicast device to device; host to device
multicast never arrives on some access points, and `239.255.x.x` groups are unreliable even to a stock
listener. Broadcast to boards is lossy under power save, unicast is not. Flood at a fixed packet rate
(the host hits `ENOBUFS` unthrottled); 1500 packets per second survived on every board. A forced Wi-Fi
reconnect on the Pico keeps IGMP membership. Set a long loop interval from `wifi.on_connect`
(`App.set_loop_interval(20000)`), not `on_boot`, or Wi-Fi connect crawls. The `logger.log` action
rejects `%.*s`; use a lambda.
## OTA overlap and lock (#20588), OTA timeouts (#19041), deflate (#19037)
All overlap orders (a web upload during a native OTA and the reverse) must be refused; the lock release
after an interrupted web upload waits about 14 s (ESP8266) or 7 s (ESP-IDF) for disconnect detection. A
refused web client on `web_server_idf` never gets the busy reply; the connection drops at reboot.
`pkill -f "a\|b"` does not alternate; use two pkills.
## DHCP lease cache (#17767)
Cycle 20 s awake, `deep_sleep.enter 30s`, wake, 20 s, `App.safe_reboot()`, logging `millis()` at the
`wifi.connected` condition. An RTS hardware reset must fall back to DHCP; a software reboot and a deep
sleep wake must apply the cache. `esphome run --no-logs` silently skipped the upload on one board; use
`upload` then `logs`.
## HDMI-CEC two-board wire (#9189)
Two boards on one wire: promiscuous mode, a 0x8F request answered with 0x90 0x00, a `destination:`
filter on the reply handler, template buttons for each target plus lambdas calling
`send(source, dest, span)` as another initiator. Healthy numbers: device-side turnaround 15 to 22 ms,
round trip 144 to 156 ms. Press buttons over the API; judge timings from device logs (ESP32 light sleep
lags the press).
## LD2450 polygon zones (#20154)
To make a target "leave" without anyone moving, set the radar's Zone-1 numbers over the target and
switch the zone type to `Filter` (the count drops to 0 within 0.5 s); set it back to `Disabled` after.
Writing those numbers blocks the ld2450 loop about 50 ms.
## Scheduler and loop wake A/B (#20356)
An external component whose template button arms an SDK `os_timer` (50 ms, one shot) on the ESP8266;
the callback runs in SYS context, records `millis()` and calls `set_timeout` or `defer`; the timeout
logs the elapsed time and loop count delta. Drive with an aioesphomeapi script (`subscribe_logs` plus
`button_command`). Build the same yaml twice with different names, one per tree.
## Noise handshake and OTA encryption on the ESP8266 (#19062, #19233)
Host side timing is useless on a weak link; use an on-device bench with the minimum of five runs over
two boots. Provision a key with `noise_encryption_set_key` through aioesphomeapi and upload with
`espota2.run_ota(host, 8266, None, bin, noise_psk=key)` when the key is not in the yaml.
## ESP-NOW on the ESP8266 (#14407)
The NONOS SDK only delivers broadcast ESP-NOW frames while the station is associated or a soft-AP is
up; a hidden, open, client-less soft-AP (60 s beacon, DHCP stopped) makes broadcasts arrive. In AP-only
mode the device transmits from the soft-AP MAC.
## Wi-Fi Easy Connect (DPP) enrollee (#17934)
Only an Android phone configures: Settings, network details, Add device, Share Wi-Fi (the Wi-Fi list's
QR scanner makes the phone an enrollee). Render the URI with the `qrcode` module and open the PNG.
`esp32: framework: log_level: DEBUG` plus `sdkconfig_options: CONFIG_WPA_DEBUG_PRINT: y` prints the
supplicant state machine.
## Thread border router (#18104)
See boards.md for the ESP Thread BR board. Check that the H2 answers Spinel before blaming a PR.
## nRF52840 dongle boards (#19009, #20814)
Test config: the two LEDs as inverted GPIO outputs, the button as an inverted pull-up binary sensor, a
10 s alive interval. To retest the nrfutil download path, move the cached `sdk-nrf/nrfutil` aside first.
@@ -0,0 +1,26 @@
# Proven test flows
Each flow names the pull requests it was used on, so their result tables show what to post.
One file per component, named after it:
- [`api.md`](api.md): noise handshake and OTA encryption on the ESP8266
- [`captive_portal.md`](captive_portal.md): fallback AP and /wifisave hammer
- [`cse7766.md`](cse7766.md): Sonoff S31 parser check over the API
- [`espnow.md`](espnow.md): ESP8266 broadcast receive
- [`hdmi_cec.md`](hdmi_cec.md): two-board wire
- [`ld2450.md`](ld2450.md): polygon zones
- [`libretiny.md`](libretiny.md): rtc preference emulation, BK72xx crash report on the next boot
- [`mqtt.md`](mqtt.md): broker bench, QoS, reconnect, crash log over MQTT
- [`nrf52.md`](nrf52.md): dongle board support
- [`online_image.md`](online_image.md): stalled and malformed downloads
- [`openthread.md`](openthread.md): border router on the S3 host plus H2 RCP
- [`ota.md`](ota.md): overlap and lock, timeouts, deflate
- [`rtsp_server.md`](rtsp_server.md): camera streaming on the Waveshare ESP32-S3-ETH
- [`scheduler.md`](scheduler.md): loop wake A/B
- [`udp.md`](udp.md): datagrams, multicast, sockets
- [`usb_cdc_acm.md`](usb_cdc_acm.md): nRF52840 dongle host driver
- [`wifi.md`](wifi.md): reboot_timeout with a fallback AP, DHCP lease cache, Easy Connect (DPP)
- [`zephyr.md`](zephyr.md): preferences on the nRF52840 dongle
- [`zephyr_coredump.md`](zephyr_coredump.md): crash and decode on nRF52 boards
- [`zwave_proxy.md`](zwave_proxy.md): frame checksums on a live proxy
@@ -0,0 +1,7 @@
# api
## Noise handshake and OTA encryption on the ESP8266 (#19062, #19233)
Host side timing is useless on a weak link; use an on-device bench with the minimum of five runs over
two boots. Provision a key with `noise_encryption_set_key` through aioesphomeapi and upload with
`espota2.run_ota(host, 8266, None, bin, noise_psk=key)` when the key is not in the yaml.
@@ -0,0 +1,21 @@
# captive_portal
## Captive portal `/wifisave` hammer (#20802, #20784)
Reproduces a `/wifisave` arriving during the STA cooldown, and proves a fix on every board.
Device under test: `templates/example-dut.yaml` (bogus network, `ap:` with `ap_timeout: 15s`,
`captive_portal:`). Client: `templates/example-client.yaml` on an ESP32 or a D1 mini; it joins the
device's AP, posts 15 bogus saves 3 s apart, then the real credentials. With the portal up the cooldown
is 30 s, so nearly every save lands inside one. Driver: `scripts/wifisave_hammer.sh`. Settle 40 s on
ESP boards, 90 s on the BW15 (its first attempt on the real network fails, see boards.md).
Counts that matter: `Requested WiFi Settings Change` (saves received), `Exiting cooldown early`, crash
markers, boots (`Log initialized`), and the client's `bogus save status 200` and `real save status 200`.
A healthy run: every save answered 200, one cooldown exit per save (two on the BW15, whose retry enters
the adapter restart), no panic, and the real credentials connect the device within about 10 s.
Fault injection for AP start failures: wrap `this->ap_setup_ = this->wifi_start_ap_(this->ap_)` in
`setup_ap_config_()` with an `HWTEST_AP_FAIL_COUNT` build flag
(`esphome: platformio_options: build_flags: -DHWTEST_AP_FAIL_COUNT=2`), raise the portal's "Captive
portal started" from LOGV to LOGI, and after provisioning curl the device's STA address: connection
refused proves the portal's server is gone.
@@ -0,0 +1,11 @@
# cse7766
## cse7766 on a Sonoff S31 (#20948)
`templates/s31-cse7766-dut.yaml`: `logger: baud_rate: 0` because the CSE7766 owns the UART, VERBOSE logger
with everything but cse7766 quieted, 2 s throttles, and a `globals` frame counter fed by the voltage sensor's
`on_raw_value` and logged every 10 s (about 96 frames per 10 s). OTA only, every reflash reboots the relay.
`scripts/cse7766/run_obs.sh IP SECONDS LABEL [OUTDIR] [NAME_FILTER]` subscribes to states and logs over the
API with a concurrent ping and prints the frame counter, API gaps and warnings. API state gaps of 10 to 20 s
can appear on a weak Wi-Fi path while the device counter stays flat, so judge a UART parser by the
device-side counter, not by the API cadence. With nothing plugged in, current and power read 0.
@@ -0,0 +1,7 @@
# espnow
## ESP-NOW on the ESP8266 (#14407)
The NONOS SDK only delivers broadcast ESP-NOW frames while the station is associated or a soft-AP is
up; a hidden, open, client-less soft-AP (60 s beacon, DHCP stopped) makes broadcasts arrive. In AP-only
mode the device transmits from the soft-AP MAC.
@@ -0,0 +1,9 @@
# hdmi_cec
## HDMI-CEC two-board wire (#9189)
Two boards on one wire: promiscuous mode, a 0x8F request answered with 0x90 0x00, a `destination:`
filter on the reply handler, template buttons for each target plus lambdas calling
`send(source, dest, span)` as another initiator. Healthy numbers: device-side turnaround 15 to 22 ms,
round trip 144 to 156 ms. Press buttons over the API; judge timings from device logs (ESP32 light sleep
lags the press).
@@ -0,0 +1,7 @@
# ld2450
## LD2450 polygon zones (#20154)
To make a target "leave" without anyone moving, set the radar's Zone-1 numbers over the target and
switch the zone type to `Filter` (the count drops to 0 within 0.5 s); set it back to `Disabled` after.
Writing those numbers blocks the ld2450 loop about 50 ms.
@@ -0,0 +1,58 @@
# libretiny
## LibreTiny rtc preference emulation (#21007)
The PR keeps `storage: rtc` preferences in FlashDB and deletes them on the first load or save after a boot
whose `lt_get_reboot_reason()` is not software, watchdog, crash or sleep. Only the families listed in
`_libretiny_rtc_supported()` accept the config keys; on others force the backend in with
`platformio_options: build_flags: ["-DUSE_LIBRETINY_RTC_PREFERENCES"]` after confirming the rejection.
- Harness: `scripts/libretiny_prefs/ltprefs.hpp` plus `templates/bk72xx-rtc-prefs-dut.yaml`. `ltprefs_boot()`
loads one rtc and one flash preference, a second pair that shares one key as rtc and as flash, and a third
rtc entry (so a hard reset drops three entries in one pass), logs the reason and both results, saves `100 + step`, syncs; after
a templated delay `ltprefs_exit()` bumps the flash-backed step and runs the step's action (`lt_reboot()`,
`__builtin_trap()`, `lt_reboot_wdt()` or idle). The last 8 boot reports are kept in a flash preference and
repeated in the 10 s alive line, so a boot missed by the capture (a power cut, an OTA-only board) is still
readable. Bump `run:` for fresh keys.
- A store to an unmapped address does not fault on the ARM968, the board carries on: use `__builtin_trap()`
for the crash row. The SDK crash handler reboots about 10 s later. Works on BK7238 and BK7231N.
- Hard resets: `serial_capture.py ... --reset` pulses RTS on CEN (reported as Power-On). A real power cut
needs a hand or a smart plug; the history in the alive line covers the boot the host cannot see. On the
Beken SDK a chip-enable reset and a power cut both report Power-On, there is no separate reset-pin source.
- Step 0 and step 8 wait 30 s before acting so the host can open the port (and pulse a reset) after a flash
without losing the first boot; the flash tool's own reset starts the sequence immediately otherwise.
- OTA-only board: keep the device's wifi, api and ota blocks verbatim, keep safe_mode, read logs with
`esphome logs <yaml> --device <name>.local` (the API attaches late, hence the history), and restore the
device's own config afterwards.
- Plain-build comparison: `raw_firmware.bin` differs by the timestamp and by `__LINE__` in log calls of
functions below the added code; map offsets through `objdump -h` load addresses and `addr2line` before
calling a byte difference a code change.
## BK72xx crash report on the next boot (#21012)
Harness: `scripts/libretiny_crash/bkprobe.hpp` plus `templates/bk72xx-crash-probe-dut.yaml`. A flash
preference holds the step; `on_boot` (priority -100) logs the reset reason, the SDK start type, two canaries
and hex dumps of the low SRAM, a set of marker words and the record area after `.bss`, waits 30 s on step 0
(so a capture can be opened after the flash) or 5 s, saves `step + 1`, syncs, then acts: 0 `__builtin_trap()`,
1 call into unmapped memory, 2 unmapped and unaligned reads and a store (no fault on the ARM968), 3
`lt_reboot()`. A `globals: restore_value` step counter does not work here: it is only saved from the global's
loop, which a crash 50 ms later never reaches; save an `ESPPreferenceObject` and `global_preferences->sync()`
first. Bump `run:` for fresh keys. Keep the `wifi:` block: the bug the fix addresses only shows with Wi-Fi up.
Flow: `flash_bk72xx.py` (retry up to three times), one `serial_capture.py ... --reset` capture of 150 to 230 s,
then `grep -a "libretiny.crash\|probe"`. Decode with
`arm-none-eabi-addr2line -pfiaC -e .pioenvs/<name>/raw_firmware.elf <addr...>` (the ELF is `raw_firmware.elf`
on LibreTiny), or run `scripts/libretiny_crash/decode_capture.py <yaml> <capture>` from the config directory
to feed the capture through the CLI's `LogLineProcessor`.
Expected: `*** CRASH DETECTED ON PREVIOUS BOOT ***`, `Reason: Undefined instruction`, PC on the `udf`,
LR = PC + 1, BT0..BT7 walking the lambda, `StatelessLambdaAction`, `Action::play_complex`, `DelayAction`,
`Scheduler::call`, `Application::loop`. The wild call reports where the core ended up (it executes zeros as
Thumb NOPs until it wraps into real code), not the called address. Without the fix a dev build with Wi-Fi
connected never comes back from the crash; see the BK72xx section in `boards.md`.
OTA-only board: copy the device's own config, add the probe include and the step machine (step 0 crashes
45 s after boot, later steps idle), keep its wifi, api, ota and safe_mode blocks and use the secrets file
next to that config. A LibreTiny OTA is a 1.5 MB UF2 and takes minutes on a slow link, so do not wrap the
upload in a short timeout. Read the replayed report with `esphome logs ... --device <name>.local` (an API
client gets it on subscribe) or on the board's serial log port if it has one.
@@ -0,0 +1,12 @@
# mqtt
## MQTT (#20433, #17935, #20762, #20767, #20768)
A local mosquitto with `allow_anonymous true`, `log_type all` and `log_dest file`: the broker log gives
each subscription's QoS, the device address (`New client connected from <ip> as <name>-<mac>`) and exact
subscribe counts. Kill and restart mosquitto to exercise `on_disconnect` (reason 0) and resubscription.
Keep `reboot_timeout: 0s` on `api:` and `mqtt:`. The ESP-IDF outbox swings free heap about 2 kB right
after bulk subscribe and unsubscribe; read the heap seconds later over several rounds before calling a
leak. Spawned subscriptions need about 2.5 s to reach the broker before the next publish.
`esphome logs --device MQTT` only uses the MQTT transport when the config has no `api:`. Driver pattern: a
paho script publishes and polls the serial capture file.
@@ -0,0 +1,6 @@
# nrf52
## nRF52840 dongle boards (#19009, #20814)
Test config: the two LEDs as inverted GPIO outputs, the button as an inverted pull-up binary sensor, a
10 s alive interval. To retest the nrfutil download path, move the cached `sdk-nrf/nrfutil` aside first.
@@ -0,0 +1,20 @@
# online_image
## online_image stalled downloads (#20594)
One classic ESP32, both frameworks (the ESP32 Arduino build also uses `http_request_idf.cpp`). The host
serves the images: `scripts/online_image/make_images.py <www>` writes 16x16 BMP/PNG/JPEG/QOI plus an 8 bit
palette BMP with a 1078 byte header; `scripts/online_image/oi_server.py <www> 8765 <log>` answers one
misbehaviour per path name (`ok`, `truncated`, `large_header`, `empty`, `stall` holds the socket 40 s,
`closed_early`, `slow` pauses 2.5 s, `chunked`, `chunked_cut`, `noclen`, `huge` claims 20 MB) with an ETag
and a 304 on If-None-Match. `templates/online-image-dut.yaml` has two AUTO-format images (default and
256 byte buffer), API actions `fetch`/`fetch_small` (set_url with `update: false`, then component.update)
and `refetch`/`refetch_small` (component.update only, keeps the cache headers), and a restart button.
`scripts/online_image/drive.py <device ip> <api key> <capture> http://<host ip>:8765 pr|base` runs the
case list over aioesphomeapi and judges each from the serial capture. ESP-IDF facts: a stalled server
ends with `Error downloading image: -28679` (EAGAIN) after the transport timeout, an early close with
`-1`, both before and after the PR; a body without Content-Length and without chunking reads as complete
(Size: 0) and finishes. A build without the fix hangs on the truncated, oversized header, empty, chunked PNG
and cut chunked cases and every later update logs `Image already being updated.`, so a baseline run needs
a reboot per group, the driver's 45 s cap per case, and an outer `timeout` that covers the whole list.
A case expected as `any` passes on `finished` or `error`, never on no outcome.
@@ -0,0 +1,5 @@
# openthread
## Thread border router (#18104)
See boards.md for the ESP Thread BR board. Check that the H2 answers Spinel before blaming a PR.
@@ -0,0 +1,8 @@
# ota
## OTA overlap and lock (#20588), OTA timeouts (#19041), deflate (#19037)
All overlap orders (a web upload during a native OTA and the reverse) must be refused; the lock release
after an interrupted web upload waits about 14 s (ESP8266) or 7 s (ESP-IDF) for disconnect detection. A
refused web client on `web_server_idf` never gets the busy reply; the connection drops at reboot.
`pkill -f "a\|b"` does not alternate; use two pkills.
@@ -0,0 +1,18 @@
# rtsp_server
## RTSP camera streaming on the Waveshare ESP32-S3-ETH (#17990)
`templates/rtsp-waveshare-s3-eth-dut.yaml` (OV5640 pins, GPIO8 camera power switch with the 1.2 s boot delay,
`rtsp_server`, logger VERBOSE for `rtsp_server`); swap the `wifi:` block for the W5500 `ethernet:` block when
the cable is in, and add `network: enable_high_performance: true` for the big TCP buffers. Flash over native
USB, capture the log with `serial_capture.py`, take the IP from the last `IP Address` line that is not
0.0.0.0 (or `dns-sd -Q <name>.local`). `scripts/rtsp_server/rtsp_client.py IP [cases]` covers play, udp,
pipeline, badreq, replay, getparam, teardown, third, stall, timeout; `rawrate2.py IP quiet|chatty|rtcp|bigbuf`
measures the raw receive rate; `measure.sh IP LABEL LOGFILE [OUTDIR]` runs the fixed set (ping, ffmpeg 12 s
twice, two ffmpeg clients, raw reader) and appends to `results.txt`; `cam_api.py IP` pulls one image over the
API; `ffmpeg -rtsp_transport tcp -i rtsp://IP:554/ -frames:v 30 -f image2 f%03d.jpg` plus Pillow checks
decoding. The 5760 byte default TCP send buffer makes the stream stall over Wi-Fi (about 1 fps, dozens of
stall drops per run); `enable_high_performance` gives several fps on Wi-Fi and removes the stalls on Ethernet.
Frame counts vary a lot between runs (auto exposure changes the JPEG size), so always measure twice. Home
Assistant's stream pipeline does not accept MJPEG over RTSP; its ffmpeg camera opens one RTSP session per
snapshot, so `max_connections` needs headroom, and go2rtc transcoding to H.264 is the way to a live view.
@@ -0,0 +1,8 @@
# scheduler
## Scheduler and loop wake A/B (#20356)
An external component whose template button arms an SDK `os_timer` (50 ms, one shot) on the ESP8266;
the callback runs in SYS context, records `millis()` and calls `set_timeout` or `defer`; the timeout
logs the elapsed time and loop count delta. Drive with an aioesphomeapi script (`subscribe_logs` plus
`button_command`). Build the same yaml twice with different names, one per tree.
@@ -0,0 +1,11 @@
# udp
## UDP, multicast, sockets (#20556 to #20559, #20585, #20590)
A listener on port 9 needs no privileges on macOS. Test multicast device to device; host to device
multicast never arrives on some access points, and `239.255.x.x` groups are unreliable even to a stock
listener. Broadcast to boards is lossy under power save, unicast is not. Flood at a fixed packet rate
(the host hits `ENOBUFS` unthrottled); 1500 packets per second survived on every board. A forced Wi-Fi
reconnect on the Pico keeps IGMP membership. Set a long loop interval from `wifi.on_connect`
(`App.set_loop_interval(20000)`), not `on_boot`, or Wi-Fi connect crawls. The `logger.log` action
rejects `%.*s`; use a lambda.
@@ -0,0 +1,20 @@
# usb_cdc_acm
## usb_cdc_acm on the nRF52840 dongle (#11634)
`templates/usb-cdc-acm-dongle-dut.yaml`: logger on USB_CDC port 0, `usb_cdc_acm` interface 0 disabled,
interface 1 echoing from a 20 ms lambda, 0x01-prefixed commands for burst, stream, flush, peek, line coding
and counters. Build and flash with `dfu: true` through the app port; the dongle then shows three CDC ports:
the board DTS node `cdc_acm_uart` (unused), `cdc_acm_uart0` (log) and `cdc_acm_uart1` (data), in that order.
Capture the log port with `serial_capture.py`, drive the data port with `scripts/usb_cdc_acm/drive.py
DATA_PORT` (11 cases, prints PASS/FAIL and a summary) and grep the log for `Opened`, `Closed`, `buffer full`,
`flush result`, `line coding`. Boot lines are held in the driver's 1 KB ring until the host opens the port, so
a capture started later still shows the first 1 KB of the boot and then a truncated line.
With three CDC instances the Zephyr legacy stack's `CONFIG_USB_MAX_NUM_TRANSFERS=4` starves the last instance
(each instance holds an OUT read and an IN ZLP), so interface 1 shows DTR changes but moves no data.
`scripts/usb_cdc_acm/cdc_probe.hpp` (include it, call `cdc_probe()` from an interval) reads the Zephyr driver
directly to tell a dead endpoint from a component bug. Raising the transfer pool to 8, or disabling the
board's `cdc_acm_uart` node, fixes it. Design limit of the driver: `fifo_fill` holds `tx_ready` until the
write callback and `write_array` drops when the ring is full, so a 64 KiB full-duplex echo loses one 64 byte
tail on every buffer size.
@@ -0,0 +1,25 @@
# wifi
## `reboot_timeout` with a fallback AP (#9958)
Only one board runs the AP at a time, or stations join the wrong one. AP probe: bogus network,
`ap: {ssid, password, ap_timeout: 5s}`, `reboot_timeout: 60s`, `logger: level: VERBOSE` (ESP32 and
ESP8266 log `AP client connected MAC=` at verbose), `on_boot: logger.log: BOOT`. Station probes:
`reboot_timeout: 0s`, `interval: 150s` running `wifi.disable`, `delay: 20s`, `wifi.enable` (60 s cycles
are too short to cover the due moment). Pass pattern: "Can't connect; rebooting" exactly 60 s after boot
with no station joined, no reboot for minutes while one is, then a reboot within a second of the last
disconnect. The BW15 restarts its AP on every STA retry and cannot be tested this way.
## DHCP lease cache (#17767)
Cycle 20 s awake, `deep_sleep.enter 30s`, wake, 20 s, `App.safe_reboot()`, logging `millis()` at the
`wifi.connected` condition. An RTS hardware reset must fall back to DHCP; a software reboot and a deep
sleep wake must apply the cache. `esphome run --no-logs` silently skipped the upload on one board; use
`upload` then `logs`.
## Wi-Fi Easy Connect (DPP) enrollee (#17934)
Only an Android phone configures: Settings, network details, Add device, Share Wi-Fi (the Wi-Fi list's
QR scanner makes the phone an enrollee). Render the URI with the `qrcode` module and open the PNG.
`esp32: framework: log_level: DEBUG` plus `sdkconfig_options: CONFIG_WPA_DEBUG_PRINT: y` prints the
supplicant state machine.
@@ -0,0 +1,10 @@
# zephyr
## Zephyr preferences on the nRF52840 dongle (#20810)
`templates/nrf52-preferences-dut.yaml`: logger on USB_CDC at VERBOSE, `flash_write_interval: 5s`, one
`globals` with `restore_value` incremented every 15 s. Give each build its own global id so each first boot
sees a never-saved key (settings survive a reflash). Flow per build: flash through the log port, 40 s capture
of the first boot, then start a second capture and reboot with a 2001 baud touch on the dongle's other CDC
port (the dfu rate callback is global, and the capture holds the log port); `serial_capture.py` reopens the
log port after the USB re-enumeration and gets the buffered boot lines (`load setting`, `Loaded N settings`).
@@ -0,0 +1,17 @@
# zephyr_coredump
## zephyr_coredump crash and decode on nRF52 boards (#13122)
`templates/nrf52-coredump-crash-dut.yaml` (dongle; change `board:` to `adafruit_itsybitsy_nrf52840` for the
ItsyBitsy): `zephyr_coredump:`, `dfu: true`, a `restore_value` global keyed by the `run` substitution
(settings survive a reflash, so bump `run` for every new crash), and an `on_boot` sequence that sets the
flag, yields 3 s so the globals loop saves it (a blocking lambda never gets there and the board crash-loops),
syncs preferences, then writes to 0xDEADBEEF. Flash, keep every port closed for 80 s, then
`scripts/zephyr_coredump/flow.sh LABEL PORT ELF [OUTDIR]` does the three opens (3 s cut, 100 s full print,
20 s reopen) and calls `decode.sh`, which keeps the first complete `#CD:BEGIN#..END#` block, runs Zephyr's
`coredump_serial_log_parser.py` and `coredump_gdbserver.py` from the sdk-nrf ESPHome installed and prints
`bt` and `info threads` with its `arm-zephyr-eabi-gdb`. The ELF is `.pioenvs/<name>/zephyr/zephyr/zephyr.elf`.
Expect about 9 KB dumps, 140 to 150 `#CD:` lines, a 673 ms loop warning for the erase, and `Has stored
dump: 0` on the reopen. Check `compiled on` in every capture: another session flashing the same board
mid-test looks exactly like a failed test otherwise. Boards without a network cannot exercise the API and
MQTT delivery paths of the component; say so in the results.
@@ -0,0 +1,23 @@
# zwave_proxy
## zwave_proxy frame checksums on a live proxy (#20922)
A Z-Wave proxy (ESP32-S3 with the module on `usb_uart`) is a live device: flash it over OTA and read logs
over the API, there is no serial console because the native USB is the host port for the module, so the
boot-time `GET_NETWORK_IDS` exchange is never captured. Home Assistant's Z-Wave JS holds the single API
subscription (it polls `0x3B` every 30 s and resubscribes after every reboot), so do not subscribe from a
script, and keep the device `name:` so it stays on Z-Wave JS; give each build its own scratch directory
instead.
1. `logger: level: VERY_VERBOSE` with `logs: zwave_proxy: VERY_VERBOSE` and DEBUG for the rest: the
`Sending:`, `Received frame:` and `CHECKSUM Received ... Calculated` lines are `ESP_LOGVV`, which needs
the compile-time level.
2. `md5` the two `firmware.ota.bin` files to be sure the A and B builds differ, then `esphome upload
--device <name>.local` and a `timeout 330 esphome logs --device <name>.local` capture started right after
the OTA to catch the resubscribe traffic.
3. `scripts/zwave_proxy/verify_checksums.py <captures>` recomputes the XOR (init 0xFF, LENGTH..payload) over
every logged frame and counts bad checksums and ACK/NAK sent and received; `scripts/zwave_proxy/devinfo.py
<name>.local` prints the firmware version, compile time and the Home ID from the API (a Home ID proves the
device-built `GET_NETWORK_IDS` frame was accepted at boot).
4. Pre-existing: one `zwave_proxy took a long time for an operation` warning on the first frame after the
log client connects, caused by VERY_VERBOSE logging; it appears on dev too.
@@ -0,0 +1,71 @@
"""Subscribe to states and logs of a device over the API for N seconds and summarize its float sensors.
usage: observe.py IP SECONDS OUTFILE [NAME_FILTER]
Every float sensor state whose entity name contains NAME_FILTER (default: all) is appended to OUTFILE
with a relative timestamp, log lines are appended with a LOG prefix, and a one-line summary per sensor
(count, min, max, last) plus the warning and error count is printed at the end.
"""
import asyncio
import pathlib
import re
import sys
import time
from aioesphomeapi import APIClient, LogLevel
IP, SECS, OUT = sys.argv[1], float(sys.argv[2]), sys.argv[3]
FILTER = sys.argv[4] if len(sys.argv) > 4 else ""
# ESPHome LOG_SECRET span; one cut off before its end marker is secret to the end of the line
SECRET = re.compile(r"\x1b\[8m.*?(?:\x1b\[28m|$)", re.DOTALL)
ANSI = re.compile(r"\x1b\[[0-9;]*m")
async def main() -> None:
c = APIClient(IP, 6053, None)
await c.connect(login=True)
di = await c.device_info()
ents, _ = await c.list_entities_services()
names = {e.key: e.name for e in ents}
readings: dict[str, list[float]] = {}
warnings: list[str] = []
t0 = time.monotonic()
with pathlib.Path(OUT).open("w", encoding="utf-8") as out:
out.write(
f"device {di.name} version {di.esphome_version} compiled {di.compilation_time}\n"
)
def on_state(st) -> None:
n = names.get(st.key, str(st.key))
if FILTER in n and hasattr(st, "state") and isinstance(st.state, float):
readings.setdefault(n, []).append(st.state)
out.write(f"[{time.monotonic() - t0:6.1f}] {n}: {st.state}\n")
out.flush()
def on_log(msg) -> None:
line = (
msg.message.decode(errors="replace")
if isinstance(msg.message, bytes)
else msg.message
)
line = ANSI.sub("", SECRET.sub("<secret>", line))
out.write(f"[{time.monotonic() - t0:6.1f}] LOG {line}\n")
out.flush()
if "[W]" in line or "[E]" in line or "Invalid" in line:
warnings.append(line)
c.subscribe_states(on_state)
c.subscribe_logs(on_log, log_level=LogLevel.LOG_LEVEL_VERBOSE)
await asyncio.sleep(SECS)
await c.disconnect()
summary = "; ".join(
f"{k.split()[-1]} n={len(v)} min={min(v):.3f} max={max(v):.3f} last={v[-1]:.3f}"
for k, v in sorted(readings.items())
)
print(f"{di.compilation_time}: {summary}; warnings/errors {len(warnings)}")
for w in warnings[:5]:
print(" ", w)
asyncio.run(main())
+27
View File
@@ -0,0 +1,27 @@
#!/bin/zsh
# usage: run_obs.sh IP SECONDS LABEL [OUTDIR] [NAME_FILTER]
# Observes states and logs over the API with a concurrent ping and prints a summary: the device-side
# "frames parsed" counter per 10 s, gaps over 5 s between voltage updates seen by the API, and warnings.
# Needs a python with aioesphomeapi (PYTHON, default python3).
D=${0:A:h}
IP=$1; SECS=$2; L=$3; S=${4:-.}; F=${5:-}
PY=${PYTHON:-python3}
setopt pipefail
ping -i 1 -c $SECS $IP > $S/ping-$L.txt 2>&1 &
P=$!
if ! timeout $((SECS + 40)) $PY $D/observe.py $IP $SECS $S/obs-$L.txt "$F" 2>&1 | sed "s/$IP/<ip>/g"; then
echo "observe.py failed or timed out, no summary"; wait $P; exit 1
fi
wait $P
echo "ping: $(tail -2 $S/ping-$L.txt | head -1 | sed "s/$IP/<ip>/g") $(tail -1 $S/ping-$L.txt)"
python3 - "$S/obs-$L.txt" <<'PY'
import re, sys
txt = open(sys.argv[1]).read()
fr = [int(m.group(1)) for m in re.finditer(r"frames parsed: (\d+) in 10 s", txt)]
print(f"device frame counter per 10 s: n={len(fr)} min={min(fr) if fr else None} max={max(fr) if fr else None} mean={sum(fr)/len(fr) if fr else 0:.1f}")
ts = [float(m.group(1)) for m in re.finditer(r"^\[\s*([0-9.]+)\] .*Voltage: ", txt, re.M)]
gaps = [round(b - a, 1) for a, b in zip(ts, ts[1:]) if b - a > 5]
print(f"API voltage updates {len(ts)}, gaps over 5 s: {gaps}")
warn = [l for l in txt.splitlines() if "LOG" in l and ("[W]" in l or "[E]" in l or "Invalid" in l)]
print(f"warnings/errors in log: {len(warn)}", *warn[:3], sep="\n ")
PY
@@ -0,0 +1,116 @@
// BK72xx crash probe: forces one crash per step and shows what the SDK and the bootloader leave behind.
// Steps: 0 undefined instruction, 1 call into unmapped memory, 2 unmapped and unaligned accesses (no fault),
// 3 lt_reboot(). The step lives in a flash preference saved and synced before the crash.
#pragma once
#include <Arduino.h>
#include <FreeRTOS.h>
#include <task.h>
#include <cstdint>
#include "esphome/core/hal.h"
#include "esphome/core/log.h"
#include "esphome/core/preferences.h"
extern "C" unsigned bk_misc_get_start_type(void);
extern "C" char _bss_end[];
static constexpr uint32_t PROBE_CANARY_LOW = 0x004000F8; // free word in the SDK-reserved low SRAM
static constexpr uint32_t PROBE_CANARY_ITCM = 0x003F3FF0; // top of ITCM
static char bkprobe_report[112];
static constexpr uint32_t PROBE_MARKS[] = {0x40A1B0, 0x40B000, 0x40C000, 0x40E000, 0x410000, 0x414000, 0x418000,
0x41C000, 0x420000, 0x428000, 0x430000, 0x438000, 0x43C000, 0x43E000,
0x43F000, 0x43F800, 0x43FC00, 0x43FE00, 0x43FFF0};
static constexpr size_t PROBE_MARK_COUNT = sizeof(PROBE_MARKS) / sizeof(PROBE_MARKS[0]);
static inline uint32_t rd(uint32_t a) { return *(volatile uint32_t *) a; }
inline esphome::ESPPreferenceObject bkprobe_pref(uint32_t run) {
return esphome::global_preferences->make_preference<int>(0xBC000000u + run);
}
inline int bkprobe_boot(uint32_t run) {
using namespace esphome;
int step = 0;
bkprobe_pref(run).load(&step);
unsigned st = bk_misc_get_start_type();
snprintf(bkprobe_report, sizeof(bkprobe_report), "step=%d reason=%s(%d) start_type=%u low=%08lx itcm=%08lx top=%08lx",
step, lt_get_reboot_reason_name(lt_get_reboot_reason()), (int) lt_get_reboot_reason(), st,
(unsigned long) rd(PROBE_CANARY_LOW), (unsigned long) rd(PROBE_CANARY_ITCM),
(unsigned long) rd(0x0043FFF0u));
ESP_LOGI("probe", "%s", bkprobe_report);
for (uint32_t a = 0x400000; a < 0x400100; a += 32) {
ESP_LOGI("probe", "%06lx: %08lx %08lx %08lx %08lx %08lx %08lx %08lx %08lx", (unsigned long) a,
(unsigned long) rd(a), (unsigned long) rd(a + 4), (unsigned long) rd(a + 8), (unsigned long) rd(a + 12),
(unsigned long) rd(a + 16), (unsigned long) rd(a + 20), (unsigned long) rd(a + 24),
(unsigned long) rd(a + 28));
}
for (size_t i = 0; i < PROBE_MARK_COUNT; i += 7) {
char line[120];
int n = 0;
for (size_t j = i; j < i + 7 && j < PROBE_MARK_COUNT; j++)
n += snprintf(line + n, sizeof(line) - n, " %06lx=%08lx", (unsigned long) PROBE_MARKS[j],
(unsigned long) rd(PROBE_MARKS[j]));
ESP_LOGI("probe", "marks%s", line);
}
uint32_t rec = ((uint32_t) _bss_end + 7) & ~7u;
for (uint32_t a = rec; a < rec + 64; a += 32) {
ESP_LOGI("probe", "rec %06lx: %08lx %08lx %08lx %08lx %08lx %08lx %08lx %08lx", (unsigned long) a,
(unsigned long) rd(a), (unsigned long) rd(a + 4), (unsigned long) rd(a + 8), (unsigned long) rd(a + 12),
(unsigned long) rd(a + 16), (unsigned long) rd(a + 20), (unsigned long) rd(a + 24),
(unsigned long) rd(a + 28));
}
*(volatile uint32_t *) PROBE_CANARY_LOW = 0xC0DE0000u | (uint32_t) step;
*(volatile uint32_t *) PROBE_CANARY_ITCM = 0x17C40000u | (uint32_t) step;
return step;
}
__attribute__((noinline)) void bkprobe_data_abort() {
using namespace esphome;
volatile uint32_t v = rd(0xFFFFFFF0);
ESP_LOGI("probe", "read 0xFFFFFFF0 -> %08lx (no fault)", (unsigned long) v);
uint32_t r;
__asm__ volatile("ldr %0, [%1]" : "=r"(r) : "r"(0x00400101) : "memory");
ESP_LOGI("probe", "unaligned ldr 0x400101 -> %08lx (no fault)", (unsigned long) r);
*(volatile uint32_t *) 0xFFFFFFF0 = 1;
ESP_LOGI("probe", "store 0xFFFFFFF0 (no fault)");
}
// act is the step number before it was bumped
inline void bkprobe_act(uint32_t run, int act) {
using namespace esphome;
int next = act + 1;
bkprobe_pref(run).save(&next);
global_preferences->sync();
ESP_LOGI("probe", "acting on step %d (saved %d)", act, next);
// mark candidate storage areas so the next boot shows what survives
uint32_t rec = ((uint32_t) _bss_end + 7) & ~7u;
for (uint32_t i = 0; i < 8; i++)
*(volatile uint32_t *) (rec + 32 + i * 4) = 0xA5A50000u | i; // second half of the record area
for (size_t i = 0; i < PROBE_MARK_COUNT; i++)
*(volatile uint32_t *) PROBE_MARKS[i] = 0x70700000u | (uint32_t) i;
ESP_LOGI("probe", "ramtop before: %08lx", (unsigned long) rd(0x0043FFF0u));
vTaskDelay(pdMS_TO_TICKS(50));
switch (act) {
case 0:
__builtin_trap(); // undefined instruction
break;
case 1: {
void (*fn)() = (void (*)()) 0xFFFFFFF1; // prefetch abort (unmapped, Thumb)
fn();
ESP_LOGI("probe", "call 0xFFFFFFF1 returned (no fault)");
break;
}
case 2:
bkprobe_data_abort();
break;
case 3:
lt_reboot();
break;
default:
break;
}
}
inline void bkprobe_alive(int step) {
using namespace esphome;
ESP_LOGI("probe", "alive step=%d uptime=%lus boot: %s", step, (unsigned long) (millis() / 1000), bkprobe_report);
}
@@ -0,0 +1,25 @@
"""Feed a serial capture through the CLI's stacktrace decoder for a build.
usage: decode_capture.py CONFIG_YAML CAPTURE
Run from the directory that holds the config (its .esphome build is used), with the esphome checkout that
built it on PYTHONPATH. Lines of the crash report are echoed, decoded frames follow them.
"""
import logging
from pathlib import Path
import sys
from esphome.config import read_config
from esphome.core import CORE
from esphome.stacktrace import LogLineProcessor
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
yaml, capture = Path(sys.argv[1]), Path(sys.argv[2])
CORE.config_path = yaml
config = read_config({})
proc = LogLineProcessor(config, CORE.target_platform)
for line in capture.read_text(encoding="utf-8", errors="replace").splitlines():
if "libretiny.crash" in line:
print(line[:120])
proc.process_line(line)
@@ -0,0 +1,155 @@
#pragma once
// BK7238 bench harness for esphome#21007: one rtc and one flash preference, a step counter in flash,
// and an exit action per step so the board drives its own soft reboot, crash and watchdog reboot.
#include <libretiny.h>
#include <cstring>
#include <FreeRTOS.h>
#include <task.h>
#include "esphome/core/hal.h"
#include "esphome/core/log.h"
#include "esphome/core/preferences.h"
using namespace esphome;
// Boot report kept for the alive line, so a boot the host could not capture is still visible
static char ltprefs_report[128];
// Last boots' reports, kept in a flash preference so boots the host never saw stay readable
struct LtprefsHistory {
char line[8][128];
};
static LtprefsHistory ltprefs_hist;
inline ESPPreferenceObject ltprefs_step_pref(uint32_t run) {
return global_preferences->make_preference<int>(0x3B210070u ^ run, true);
}
// Loads the step and both preferences, logs them, saves this step's value.
// Returns the exit action: 0 idle, 1 lt_reboot, 2 crash, 3 watchdog.
inline int ltprefs_boot(int &step, uint32_t run) {
auto reason = lt_get_reboot_reason();
step = 0;
ltprefs_step_pref(run).load(&step);
auto rtcp = global_preferences->make_preference<uint32_t>(0x1B210070u ^ run, false);
auto flp = global_preferences->make_preference<uint32_t>(0x2B210070u ^ run, true);
uint32_t rv = 0, fv = 0;
// a second pair sharing one key as rtc and as flash, and a third rtc entry, so a hard reset drops three
auto rtcp2 = global_preferences->make_preference<uint32_t>(0x5B210070u ^ run, false);
auto flp2 = global_preferences->make_preference<uint32_t>(0x5B210070u ^ run, true);
auto rtcp3 = global_preferences->make_preference<uint32_t>(0x6B210070u ^ run, false);
uint32_t rv2 = 0, fv2 = 0, rv3 = 0;
bool rok = rtcp.load(&rv);
bool fok = flp.load(&fv);
bool rok2 = rtcp2.load(&rv2);
bool fok2 = flp2.load(&fv2);
bool rok3 = rtcp3.load(&rv3);
snprintf(ltprefs_report, sizeof(ltprefs_report),
"step=%d reason=%s(%d) rtc=%s%u flash=%s%u sharedkey rtc=%s%u flash=%s%u rtc3=%s%u", step,
lt_get_reboot_reason_name(reason), (int) reason, rok ? "" : "none ", rv, fok ? "" : "none ", fv,
rok2 ? "" : "none ", rv2, fok2 ? "" : "none ", fv2, rok3 ? "" : "none ", rv3);
ESP_LOGI("lttest", "%s", ltprefs_report);
auto histp = global_preferences->make_preference<LtprefsHistory>(0x4B210070u ^ run, true);
if (!histp.load(&ltprefs_hist))
memset(&ltprefs_hist, 0, sizeof(ltprefs_hist));
memmove(&ltprefs_hist.line[1], &ltprefs_hist.line[0], sizeof(ltprefs_hist.line) - sizeof(ltprefs_hist.line[0]));
strncpy(ltprefs_hist.line[0], ltprefs_report, sizeof(ltprefs_hist.line[0]) - 1);
ltprefs_hist.line[0][sizeof(ltprefs_hist.line[0]) - 1] = 0;
if (!histp.save(&ltprefs_hist)) {
ESP_LOGW("lttest", "history save failed");
}
int act = 0;
bool save = true;
switch (step) {
case 0:
act = 1;
break; // soft reboot
case 1:
act = 2;
break; // crash
case 2:
act = 3;
break; // watchdog
case 3:
act = 0;
break; // idle: chip-enable reset from the host
case 4:
act = 1;
break; // soft reboot after the hard reset
case 5:
act = 0;
break; // idle: chip-enable reset from the host
case 6:
act = 1;
save = false;
break; // no save, soft reboot: deleted, not skipped?
case 7:
act = 0;
break; // idle
case 8:
act = 2;
break; // crash
case 9:
act = 0;
break; // idle
case 10:
act = 0;
break; // idle: power cut by the user
default:
act = 0;
save = false;
break;
}
if (save) {
uint32_t v = 100 + step, v2r = 200 + step, v2f = 300 + step, v3 = 400 + step;
bool srtc = rtcp.save(&v);
bool sfl = flp.save(&v);
bool srtc2 = rtcp2.save(&v2r);
bool sfl2 = flp2.save(&v2f);
bool srtc3 = rtcp3.save(&v3);
bool synced = global_preferences->sync();
ESP_LOGI("lttest", "save %u/%u/%u/%u: rtc=%d flash=%d sharedkey rtc=%d flash=%d rtc3=%d sync=%d", v, v2r, v2f, v3,
srtc, sfl, srtc2, sfl2, srtc3, synced);
}
ESP_LOGI("lttest", "exit action=%d", act);
return act;
}
inline void ltprefs_exit(int &step, uint32_t run, int act) {
step++;
bool saved = ltprefs_step_pref(run).save(&step);
bool synced = global_preferences->sync();
ESP_LOGI("lttest", "step %d save=%d sync=%d", step, saved, synced);
if (!saved || !synced) {
ESP_LOGE("lttest", "step not stored, staying idle instead of repeating the step");
return;
}
vTaskDelay(pdMS_TO_TICKS(200));
switch (act) {
case 1:
ESP_LOGW("lttest", "soft reboot now");
vTaskDelay(pdMS_TO_TICKS(50));
lt_reboot();
break;
case 2:
ESP_LOGW("lttest", "crashing now");
vTaskDelay(pdMS_TO_TICKS(50));
__builtin_trap(); // undefined instruction: a plain bad store does not fault on the ARM968
break;
case 3:
ESP_LOGW("lttest", "watchdog reboot now");
vTaskDelay(pdMS_TO_TICKS(50));
lt_reboot_wdt();
break;
default:
ESP_LOGW("lttest", "idle, waiting for an external reset");
break;
}
}
inline void ltprefs_alive(int step, int act) {
ESP_LOGI("lttest", "alive step=%d act=%d boot: %s", step, act, ltprefs_report);
for (int i = 7; i >= 0; i--) {
if (ltprefs_hist.line[i][0] != 0) {
ESP_LOGI("lttest", " history[-%d]: %s", i, ltprefs_hist.line[i]);
}
}
}
@@ -0,0 +1,179 @@
"""Drive the online_image cases over the native API and judge each from the serial capture."""
import asyncio
import pathlib
import re
import sys
import time
from aioesphomeapi import (
APIClient,
APIConnectionError,
InvalidEncryptionKeyAPIError,
RequiresEncryptionAPIError,
)
HOST, KEY, CAPTURE, BASE, MODE = sys.argv[1:6]
CAP_S = 45
CASES = [
# (label, action, path, expected outcome)
("ok bmp", "fetch", "ok.bmp", "finished"),
("ok png", "fetch", "ok.png", "finished"),
("ok jpg", "fetch", "ok.jpg", "finished"),
("ok qoi", "fetch", "ok.qoi", "finished"),
("truncated bmp", "fetch", "truncated.bmp", "error"),
("ok bmp after error", "fetch", "ok.bmp", "finished"),
("large header, 256 byte buffer", "fetch_small", "large_header.bmp", "error"),
("ok bmp, 256 byte buffer", "fetch_small", "ok.bmp", "finished"),
("empty body", "fetch", "empty.bmp", "error"),
("stalled connection", "fetch", "stall.bmp", "error"),
("closed early", "fetch", "closed_early.bmp", "error"),
("slow but alive", "fetch", "slow.bmp", "finished"),
("chunked bmp", "fetch", "chunked.bmp", "finished"),
("chunked png", "fetch", "chunked.png", "finished"),
("chunked qoi", "fetch", "chunked.qoi", "finished"),
("chunked bmp cut", "fetch", "chunked_cut.bmp", "error"),
("chunked png cut", "fetch", "chunked_cut.png", "any"),
("no content-length", "fetch", "noclen.bmp", "any"),
("huge jpeg claim", "fetch", "huge.jpg", "error"),
("ok jpg after huge", "fetch", "ok.jpg", "finished"),
("ok bmp again", "fetch", "ok.bmp", "finished"),
("ok bmp refetch: 304", "refetch", "", "finished"),
("ok png", "fetch", "ok.png", "finished"),
("ok png refetch: 304", "refetch", "", "finished"),
("chunked bmp", "fetch", "chunked.bmp", "finished"),
("chunked bmp refetch: 304", "refetch", "", "finished"),
("chunked png", "fetch", "chunked.png", "finished"),
("chunked png refetch: not cached", "refetch", "", "finished"),
("chunked png cut", "fetch", "chunked_cut.png", "any"),
("chunked png cut refetch: not cached", "refetch", "", "any"),
]
if MODE == "base":
# Each stuck case is followed by a fetch that shows the symptom, then a reboot.
CASES = [
("ok bmp", "fetch", "ok.bmp", "finished"),
("truncated bmp", "fetch", "truncated.bmp", "error"),
("ok bmp after truncated", "fetch", "ok.bmp", "finished"),
("REBOOT", "", "", ""),
("large header, 256 byte buffer", "fetch_small", "large_header.bmp", "error"),
("ok bmp, 256 byte buffer", "fetch_small", "ok.bmp", "finished"),
("REBOOT", "", "", ""),
("empty body", "fetch", "empty.bmp", "error"),
("ok bmp after empty", "fetch", "ok.bmp", "finished"),
("REBOOT", "", "", ""),
("stalled connection", "fetch", "stall.bmp", "error"),
("ok bmp after stall", "fetch", "ok.bmp", "finished"),
("REBOOT", "", "", ""),
("closed early", "fetch", "closed_early.bmp", "error"),
("ok bmp after closed early", "fetch", "ok.bmp", "finished"),
("REBOOT", "", "", ""),
("chunked png", "fetch", "chunked.png", "finished"),
("ok bmp after chunked png", "fetch", "ok.bmp", "finished"),
("REBOOT", "", "", ""),
("chunked bmp cut", "fetch", "chunked_cut.bmp", "error"),
("ok bmp after chunked cut", "fetch", "ok.bmp", "finished"),
("REBOOT", "", "", ""),
("huge jpeg claim", "fetch", "huge.jpg", "error"),
("ok jpg after huge", "fetch", "ok.jpg", "finished"),
]
OUTCOME = re.compile(r"download (finished|error): (img_small|img)")
INTEREST = re.compile(
r"\[(E|W)\]\[online_image|Image already being updated|download (finished|error)|Downloading image|Updating image|Image fully downloaded|HTTP 304|allocat|Panic|abort\(\)|Guru|Backtrace|rst:0x",
re.IGNORECASE,
)
def tail(path: str, offset: int) -> tuple[list[str], int]:
with pathlib.Path(path).open("rb") as f:
f.seek(offset)
data = f.read()
return data.decode("utf-8", "replace").splitlines(), offset + len(data)
async def connect() -> tuple[APIClient, dict, int]:
for _ in range(60):
try:
c = APIClient(HOST, 6053, None, noise_psk=KEY)
await c.connect(login=True)
ents, services = await c.list_entities_services()
except (InvalidEncryptionKeyAPIError, RequiresEncryptionAPIError) as exc:
raise SystemExit(f"API key problem, not retrying: {exc}") from exc
except (APIConnectionError, OSError) as exc:
# only connection trouble is retried; a wrong key or a missing button must fail at once
print(f"connect: {exc}", flush=True)
await asyncio.sleep(3)
continue
btn = next(
(e.key for e in ents if getattr(e, "object_id", "") == "reboot"), None
)
if btn is None:
raise SystemExit("the config has no reboot button")
return c, {s.name: s for s in services}, btn
raise SystemExit("API never connected")
async def main() -> None:
client, actions, btn = await connect()
with pathlib.Path(CAPTURE).open("rb") as f:
f.seek(0, 2)
offset = f.tell()
rows = []
for label, action, path, expect in CASES:
if label == "REBOOT":
client.button_command(btn)
await asyncio.sleep(1)
await client.disconnect()
await asyncio.sleep(8)
client, actions, btn = await connect()
_, offset = tail(CAPTURE, offset)
print("--- rebooted", flush=True)
continue
img = "img_small" if action.endswith("_small") else "img"
t0 = time.monotonic()
args = {} if action.startswith("refetch") else {"url": f"{BASE}/{path}"}
await client.execute_service(actions[action], args)
outcome, notes = "no outcome", []
while time.monotonic() - t0 < CAP_S:
await asyncio.sleep(0.5)
lines, offset = tail(CAPTURE, offset)
done = False
for line in lines:
if INTEREST.search(line):
notes.append(line.strip())
m = OUTCOME.search(line)
if m and m.group(2) == img:
outcome = m.group(1)
done = True
if done:
break
took = time.monotonic() - t0
if outcome == "no outcome":
took = CAP_S
# drain a little more so the "already being updated" line lands
await asyncio.sleep(1)
lines, offset = tail(CAPTURE, offset)
notes += [line.strip() for line in lines if INTEREST.search(line)]
ok = outcome == expect or (expect == "any" and outcome != "no outcome")
rows.append((label, path, expect, outcome, took, ok, notes))
print(
f"{'PASS' if ok else 'FAIL'} {label:32s} {path:18s} expected {expect:9s} got {outcome:10s} {took:5.1f}s",
flush=True,
)
for n in notes:
print(f" {n}", flush=True)
await asyncio.sleep(1.5)
await client.disconnect()
print(
"\nSUMMARY",
"all expectations met"
if all(r[5] for r in rows)
else "MISMATCHES: " + ", ".join(r[0] for r in rows if not r[5]),
flush=True,
)
if not all(r[5] for r in rows):
sys.exit(1)
asyncio.run(main())
@@ -0,0 +1,38 @@
"""Build the small test images the HTTP server serves."""
from pathlib import Path
import struct
import sys
from PIL import Image
out = Path(sys.argv[1])
out.mkdir(exist_ok=True)
w = h = 16
im = Image.new("RGB", (w, h))
px = im.load()
for y in range(h):
for x in range(w):
px[x, y] = (x * 16, y * 16, (x + y) * 8)
im.save(out / "ok.bmp", "BMP")
im.save(out / "ok.png", "PNG")
im.save(out / "ok.jpg", "JPEG", quality=90)
# QOI with only QOI_OP_RGB codes: valid and trivially encoded.
qoi = b"qoif" + struct.pack(">IIBB", w, h, 3, 0)
for y in range(h):
for x in range(w):
qoi += bytes([0xFE, *px[x, y]])
qoi += bytes(7) + b"\x01"
(out / "ok.qoi").write_bytes(qoi)
# 8x8 8-bit palette BMP: 1078 byte header, pixels start beyond a 256 byte buffer.
pw = ph = 8
data_offset = 14 + 40 + 256 * 4
file_size = data_offset + pw * ph
hdr = b"BM" + struct.pack("<IHHI", file_size, 0, 0, data_offset)
dib = struct.pack("<IiiHHIIiiII", 40, pw, ph, 1, 8, 0, pw * ph, 0, 0, 256, 0)
(out / "large_header.bmp").write_bytes(hdr + dib + bytes(256 * 4) + bytes(pw * ph))
for p in sorted(out.iterdir()):
print(p.name, p.stat().st_size)
@@ -0,0 +1,142 @@
"""HTTP server with one misbehaviour per path, for the online_image stalled-download test."""
import asyncio
from pathlib import Path
import sys
import time
WWW = Path(sys.argv[1])
PORT = int(sys.argv[2])
LOG = Path(sys.argv[3]).open("a", encoding="utf-8", buffering=1) # noqa: SIM115
MIME = {"bmp": "image/bmp", "png": "image/png", "jpg": "image/jpeg", "qoi": "image/qoi"}
IMG = {n: (WWW / f"ok.{n}").read_bytes() for n in MIME}
BMP = IMG["bmp"]
BMP_HDR = 54 # 16x16 24-bit: pixel rows follow directly
LARGE_HEADER = (WWW / "large_header.bmp").read_bytes()
def log(msg: str) -> None:
LOG.write(f"{time.strftime('%H:%M:%S')} {msg}\n")
def headers(ctype: str, extra: list[bytes]) -> bytes:
lines = [
b"HTTP/1.1 200 OK",
b"Content-Type: " + ctype.encode(),
b"Connection: close",
*extra,
b"",
b"",
]
return b"\r\n".join(lines)
def etag(path: str) -> bytes:
return b'"' + path.strip("/").encode() + b'-v1"'
def chunked(body: bytes, size: int) -> bytes:
out = b""
for i in range(0, len(body), size):
part = body[i : i + size]
out += b"%x\r\n" % len(part) + part + b"\r\n"
return out + b"0\r\n\r\n"
async def respond(path: str, req: bytes, w: asyncio.StreamWriter) -> str:
name, _, ext = path.lstrip("/").rpartition(".")
ctype = MIME.get(ext, "application/octet-stream")
body = IMG.get(ext, BMP)
tag = [b"ETag: " + etag(path)]
if b"if-none-match: " + etag(path).lower() in req.lower():
w.write(
b"HTTP/1.1 304 Not Modified\r\nETag: "
+ etag(path)
+ b"\r\nConnection: close\r\n\r\n"
)
return "304 not modified"
if name == "ok":
w.write(headers(ctype, [b"Content-Length: %d" % len(body), *tag]) + body)
return f"full {len(body)} bytes"
if name == "truncated": # cut one byte into the sixth pixel, Content-Length matches
cut = BMP[: BMP_HDR + 5 * 3 + 1]
w.write(headers(ctype, [b"Content-Length: %d" % len(cut)]) + cut)
return f"truncated {len(cut)} bytes"
if name == "large_header":
w.write(
headers(ctype, [b"Content-Length: %d" % len(LARGE_HEADER)]) + LARGE_HEADER
)
return f"large header {len(LARGE_HEADER)} bytes"
if name == "empty":
w.write(headers(ctype, [b"Content-Length: 0"]))
return "empty body"
if (
name == "stall"
): # claims the full length, sends part, then holds the socket open
part = body[: BMP_HDR + 15]
w.write(headers(ctype, [b"Content-Length: %d" % len(body)]) + part)
await w.drain()
await asyncio.sleep(40)
return f"stalled after {len(part)} bytes, closed after 40 s"
if name == "closed_early": # claims the full length, sends part, closes at once
part = body[: BMP_HDR + 15]
w.write(headers(ctype, [b"Content-Length: %d" % len(body)]) + part)
return f"closed after {len(part)} of {len(body)} bytes"
if name == "slow": # pauses under the timeout, then completes
w.write(headers(ctype, [b"Content-Length: %d" % len(body)]) + body[:BMP_HDR])
await w.drain()
await asyncio.sleep(2.5)
w.write(body[BMP_HDR:])
return f"full {len(body)} bytes with a 2.5 s pause"
if name == "chunked":
w.write(
headers(ctype, [b"Transfer-Encoding: chunked", *tag]) + chunked(body, 64)
)
return f"chunked full {len(body)} bytes"
if (
name == "chunked_cut"
): # chunked, ends cleanly at a pixel boundary (or half a png)
cut = body[: BMP_HDR + 5 * 3] if ext == "bmp" else body[: len(body) // 2]
w.write(
headers(ctype, [b"Transfer-Encoding: chunked", *tag]) + chunked(cut, 64)
)
return f"chunked cut {len(cut)} of {len(body)} bytes"
if name == "noclen": # body delimited by connection close
w.write(headers(ctype, []) + body)
return f"no content-length, full {len(body)} bytes"
if name == "huge": # claims 20 MB so the JPEG buffer resize fails
w.write(headers(ctype, [b"Content-Length: 20000000"]) + body)
return f"claims 20000000, sent {len(body)} bytes"
w.write(b"HTTP/1.1 404 Not Found\r\nContent-Length: 0\r\nConnection: close\r\n\r\n")
return "404"
async def handle(r: asyncio.StreamReader, w: asyncio.StreamWriter) -> None:
peer = w.get_extra_info("peername")
try:
async with asyncio.timeout(5):
line = await r.readuntil(b"\r\n")
req = await r.readuntil(b"\r\n\r\n")
path = line.split(b" ")[1].decode()
log(
f"GET {path} from {peer}"
+ (" (If-None-Match)" if b"if-none-match" in req.lower() else "")
)
what = await respond(path, req, w)
await w.drain()
log(f" -> {path}: {what}")
except Exception as exc: # noqa: BLE001
log(f" !! {peer}: {exc!r}")
finally:
w.close()
async def main() -> None:
server = await asyncio.start_server(handle, "0.0.0.0", PORT)
log(f"listening on {PORT}")
async with server:
await server.serve_forever()
asyncio.run(main())
@@ -0,0 +1,33 @@
"""Request one camera image over the native API and report its size (direct esp32_camera check)."""
import asyncio
import sys
from aioesphomeapi import APIClient
async def main():
c = APIClient(sys.argv[1], 6053, None)
await c.connect(login=True)
ents, _ = await c.list_entities_services()
cams = [e for e in ents if type(e).__name__ == "CameraInfo"]
print("camera entities:", [e.name for e in cams])
got = asyncio.get_event_loop().create_future()
def on_state(state):
if type(state).__name__ == "CameraState" and not got.done():
got.set_result(len(state.data))
c.subscribe_states(on_state)
c.request_single_image()
try:
n = await asyncio.wait_for(got, 10)
print("single image bytes:", n)
except TimeoutError:
print("no image within 10 s")
await c.disconnect()
sys.exit(1)
await c.disconnect()
asyncio.run(main())
@@ -0,0 +1,32 @@
#!/bin/zsh
# usage: measure.sh IP LABEL LOGFILE [OUTDIR]
# Runs the fixed RTSP measurement set against one build and appends a result line to OUTDIR/results.txt:
# ping RTT, two 12 s ffmpeg runs, two concurrent ffmpeg clients, and the raw receive rate with the default
# and a 1 MiB socket buffer. LOGFILE is the device capture, used to count "stalled" drops during the run.
# A measurement that produced nothing is written as ERR and the script exits 1, so a broken run never looks
# like a measured one. Needs ffmpeg and a python with the rtsp scripts' dependencies (PYTHON, default python3).
D=${0:A:h}
IP=$1; LABEL=$2; LOG=$3; S=${4:-.}
PY=${PYTHON:-python3}
[ -f "$LOG" ] || { echo "device capture not found: $LOG"; exit 1; }
rtt=$(ping -c 10 -i 0.2 $IP 2>&1 | tail -1 | awk -F/ '{print $5}')
stall0=$(grep -a -c "stalled" $LOG)
frames() { # OUTFILE RC: the frame count of a run that finished on its own, nothing for a failed or killed one
[ "$2" = 0 ] && grep -o 'frame= *[0-9]*' $1 | tail -1 | tr -dc 0-9
}
timeout 40 ffmpeg -hide_banner -rtsp_transport tcp -i rtsp://$IP:554/ -t 12 -f null - > $S/m-f1.txt 2>&1; f1=$(frames $S/m-f1.txt $?); sleep 2
timeout 40 ffmpeg -hide_banner -rtsp_transport tcp -i rtsp://$IP:554/ -t 12 -f null - > $S/m-f2.txt 2>&1; f2=$(frames $S/m-f2.txt $?); sleep 2
timeout 40 ffmpeg -hide_banner -rtsp_transport tcp -i rtsp://$IP:554/ -t 12 -f null - > $S/m-twoA.txt 2>&1 & A=$!; sleep 1
timeout 40 ffmpeg -hide_banner -rtsp_transport tcp -i rtsp://$IP:554/ -t 12 -f null - > $S/m-twoB.txt 2>&1 & B=$!
wait $A; ra=$?; wait $B; rb=$?
fa=$(frames $S/m-twoA.txt $ra); fb=$(frames $S/m-twoB.txt $rb); sleep 2
timeout 40 $PY $D/rawrate2.py $IP quiet > $S/m-raw.txt 2>&1 && raw=$(grep -o "[0-9.]* KiB/s" $S/m-raw.txt | head -1); sleep 2
timeout 40 $PY $D/rawrate2.py $IP bigbuf > $S/m-raw2.txt 2>&1 && raw2=$(grep -o "[0-9.]* KiB/s" $S/m-raw2.txt | head -1); sleep 1
stall1=$(grep -a -c "stalled" $LOG)
bad=0
for v in rtt f1 f2 fa fb raw raw2; do
if [ -z "${(P)v}" ]; then eval "$v=ERR"; bad=1; fi
done
line="$LABEL | rtt avg ${rtt} ms | ffmpeg single 12 s: ${f1}, ${f2} frames | two clients: ${fa} + ${fb} | python raw: ${raw} (1 MiB rcvbuf ${raw2}) | stalls logged: $((stall1 - stall0))"
echo "$line" | tee -a $S/results.txt
if [ $bad = 1 ]; then echo "one or more measurements failed (ERR above)"; exit 1; fi
@@ -0,0 +1,83 @@
import contextlib
import socket
import sys
import time
IP = sys.argv[1]
mode = sys.argv[2]
secs = 8
s = socket.create_connection((IP, 554), timeout=5)
if mode == "bigbuf":
s.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 1 << 20)
cseq = 0
buf = b""
def read_response() -> bytes:
"""One complete RTSP response: headers through the blank line plus a Content-Length body."""
global buf # noqa: PLW0603
while (end := buf.find(b"\r\n\r\n")) < 0:
chunk = s.recv(4096)
if not chunk:
raise ConnectionError("closed")
buf += chunk
head = buf[:end].decode(errors="replace")
body_len = 0
for line in head.split("\r\n"):
if line.lower().startswith("content-length:"):
body_len = int(line.split(":", 1)[1])
while len(buf) < end + 4 + body_len:
chunk = s.recv(4096)
if not chunk:
raise ConnectionError("closed")
buf += chunk
resp, buf = buf[: end + 4 + body_len], buf[end + 4 + body_len :]
return resp
def req(m, extra="", wait=True):
global cseq # noqa: PLW0603
cseq += 1
url = f"rtsp://{IP}:554/trackID=0" if m == "SETUP" else f"rtsp://{IP}:554/"
s.sendall(f"{m} {url} RTSP/1.0\r\nCSeq: {cseq}\r\n{extra}\r\n".encode())
return read_response() if wait else b""
req("OPTIONS")
req("DESCRIBE", "Accept: application/sdp\r\n")
r = req("SETUP", "Transport: RTP/AVP/TCP;unicast;interleaved=0-1\r\n")
sess = (
[x for x in r.decode(errors="replace").split("\r\n") if x.startswith("Session")][0]
.split()[1]
.split(";")[0]
)
req("PLAY", f"Session: {sess}\r\n")
t0 = time.monotonic()
total = 0
chunks = 0
last_chat = t0
s.settimeout(2)
while time.monotonic() - t0 < secs:
now = time.monotonic()
if mode == "chatty" and now - last_chat > 0.2:
req("GET_PARAMETER", f"Session: {sess}\r\n", wait=False)
last_chat = now
if mode == "rtcp" and now - last_chat > 0.2:
s.sendall(b"$\x01\x00\x08" + b"\x80\xc9\x00\x01\x00\x00\x00\x01")
last_chat = now
try:
d = s.recv(65536)
except TimeoutError:
continue
if not d:
break
total += len(d)
chunks += 1
print(
f"{mode:7s}: {total / 1024 / secs:5.1f} KiB/s, {chunks} recvs, avg {total // max(chunks, 1)} B/recv"
)
with contextlib.suppress(OSError):
req("TEARDOWN", f"Session: {sess}\r\n")
s.close()
@@ -0,0 +1,345 @@
"""Raw RTSP/TCP client for the esphome rtsp_server hardware test.
usage: rtsp_client.py HOST [case ...]
cases: play, udp, pipeline, third, stall, teardown, getparam, badreq, replay, timeout
"""
import socket
import struct
import sys
import time
HOST = sys.argv[1]
PORT = 554
cases = sys.argv[2:] or [
"play",
"udp",
"pipeline",
"badreq",
"replay",
"getparam",
"teardown",
"third",
"stall",
]
results = []
def say(case, ok, detail):
results.append((case, ok, detail))
print(f"{'PASS' if ok else 'FAIL'} {case}: {detail}", flush=True)
class Client:
def __init__(self, timeout=5):
self.s = socket.create_connection((HOST, PORT), timeout=timeout)
self.cseq = 0
self.session = None
self.buf = b""
def send(self, method, extra="", url=None):
self.cseq += 1
url = (
url or f"rtsp://{HOST}:{PORT}/trackID=0"
if method == "SETUP"
else (url or f"rtsp://{HOST}:{PORT}/")
)
req = f"{method} {url} RTSP/1.0\r\nCSeq: {self.cseq}\r\n"
if self.session:
req += f"Session: {self.session}\r\n"
req += extra + "\r\n"
self.s.sendall(req.encode())
return self.cseq
def _recv(self):
chunk = self.s.recv(4096)
if not chunk:
raise ConnectionError("closed")
self.buf += chunk
def read_response(self):
# Read until a full RTSP response (headers + optional body); skip any interleaved data in between
while True:
if self.buf.startswith(b"$"):
if len(self.buf) < 4:
self._recv()
continue
ln = struct.unpack(">H", self.buf[2:4])[0]
if len(self.buf) < 4 + ln:
self._recv()
continue
self.buf = self.buf[4 + ln :]
continue
end = self.buf.find(b"\r\n\r\n")
if end < 0:
self._recv()
continue
head = self.buf[:end].decode(errors="replace")
body_len = 0
for line in head.split("\r\n"):
if line.lower().startswith("content-length:"):
body_len = int(line.split(":", 1)[1])
while len(self.buf) < end + 4 + body_len:
self._recv()
body = self.buf[end + 4 : end + 4 + body_len]
self.buf = self.buf[end + 4 + body_len :]
status = int(head.split(" ", 2)[1])
for line in head.split("\r\n"):
if line.lower().startswith("session:"):
self.session = line.split(":", 1)[1].strip().split(";")[0]
return status, head, body
def read_frames(self, seconds):
"""Collect interleaved RTP packets on channel 0 for `seconds`; return (frames, packets, seq_gaps, sizes)."""
deadline = time.monotonic() + seconds
frames = 0
packets = 0
gaps = 0
last_seq = None
cur = 0
sizes = []
self.s.settimeout(1.0)
while time.monotonic() < deadline:
try:
chunk = self.s.recv(65536)
except TimeoutError:
continue
if not chunk:
break
self.buf += chunk
while True:
if len(self.buf) < 4 or self.buf[0:1] != b"$":
if self.buf and self.buf[0:1] != b"$":
end = self.buf.find(b"\r\n\r\n")
if end < 0:
break
self.buf = self.buf[end + 4 :]
continue
break
ch = self.buf[1]
ln = struct.unpack(">H", self.buf[2:4])[0]
if len(self.buf) < 4 + ln:
break
pkt = self.buf[4 : 4 + ln]
self.buf = self.buf[4 + ln :]
if ch != 0 or len(pkt) < 12:
continue
packets += 1
seq = struct.unpack(">H", pkt[2:4])[0]
marker = pkt[1] & 0x80
if last_seq is not None and seq != ((last_seq + 1) & 0xFFFF):
gaps += 1
last_seq = seq
cur += len(pkt) - 12
if marker:
frames += 1
sizes.append(cur)
cur = 0
return frames, packets, gaps, sizes
def close(self):
self.s.close()
def setup_play(c):
c.send("OPTIONS")
st, head, _ = c.read_response()
assert st == 200, head
c.send("DESCRIBE", "Accept: application/sdp\r\n")
st, head, body = c.read_response()
assert st == 200, head
c.send("SETUP", "Transport: RTP/AVP/TCP;unicast;interleaved=0-1\r\n")
st, head, _ = c.read_response()
assert st == 200, head
c.send("PLAY")
st, head, _ = c.read_response()
assert st == 200, head
return body
for case in cases:
try:
if case == "play":
c = Client()
sdp = setup_play(c)
frames, packets, gaps, sizes = c.read_frames(10)
c.send("TEARDOWN")
c.read_response()
c.close()
ok = frames >= 50 and gaps == 0
say(
case,
ok,
f"{frames} frames in 10 s ({frames / 10:.1f} fps), {packets} RTP packets, {gaps} seq gaps, frame size min {min(sizes) if sizes else 0} max {max(sizes) if sizes else 0}; sdp has m=video {'m=video' in sdp.decode()}",
)
elif case == "udp":
c = Client()
c.send("OPTIONS")
c.read_response()
c.send("SETUP", "Transport: RTP/AVP;unicast;client_port=5000-5001\r\n")
st, head, _ = c.read_response()
c.close()
say(case, st == 461, f"UDP SETUP answered {st}")
elif case == "pipeline":
c = Client()
c.s.sendall(
f"OPTIONS rtsp://{HOST}/ RTSP/1.0\r\nCSeq: 1\r\n\r\nDESCRIBE rtsp://{HOST}/ RTSP/1.0\r\nCSeq: 2\r\nAccept: application/sdp\r\n\r\n".encode()
)
c.cseq = 2
st1, h1, _ = c.read_response()
st2, h2, b2 = c.read_response()
c.close()
cs = [x for x in (h1 + h2).split("\r\n") if x.startswith("CSeq")]
say(
case,
st1 == 200 and st2 == 200 and cs == ["CSeq: 1", "CSeq: 2"],
f"two pipelined requests answered {st1}, {st2} with {cs}",
)
elif case == "badreq":
c = Client()
c.s.sendall(b"BOGUS rtsp://x/ RTSP/1.0\r\nCSeq: 7\r\n\r\n")
st, head, _ = c.read_response()
c.close()
say(
case,
st in (400, 405, 501) and "CSeq: 7" in head,
f"unknown method answered {st}, CSeq echoed {'CSeq: 7' in head}",
)
elif case == "replay":
c = Client()
setup_play(c)
f1, _, _, _ = c.read_frames(3)
c.send("PLAY")
st, head, _ = c.read_response()
f2, _, gaps, _ = c.read_frames(3)
c.send("TEARDOWN")
c.read_response()
c.close()
say(
case,
st == 200 and f2 >= 10 and gaps == 0,
f"second PLAY answered {st}, {f1} then {f2} frames, {gaps} seq gaps after repeat PLAY",
)
elif case == "getparam":
c = Client()
setup_play(c)
c.read_frames(1)
c.send("GET_PARAMETER")
st, head, _ = c.read_response()
f, _, _, _ = c.read_frames(2)
c.send("TEARDOWN")
c.read_response()
c.close()
say(
case,
st == 200 and f > 5,
f"GET_PARAMETER while playing answered {st}, stream continued with {f} frames in 2 s",
)
elif case == "teardown":
c = Client()
setup_play(c)
c.read_frames(2)
c.send("TEARDOWN")
st, head, _ = c.read_response()
c.s.settimeout(3)
time.sleep(1.5)
try:
rest = c.s.recv(65536)
closed = rest == b""
except TimeoutError:
rest = b"x"
closed = False
except ConnectionError:
closed = True
c.close()
say(
case,
st == 200,
f"TEARDOWN answered {st}; connection closed by server afterwards: {closed}",
)
elif case == "third":
a = Client()
setup_play(a)
b = Client()
setup_play(b)
fa, _, ga, _ = a.read_frames(4)
fb, _, gb, _ = b.read_frames(4)
try:
d = Client(timeout=3)
d.send("OPTIONS")
st, _, _ = d.read_response()
third = f"third got {st}"
d.close()
except ConnectionError as e:
# refused, reset or closed by the server; a timeout or a bug propagates and fails the case
third = f"third rejected ({type(e).__name__})"
a.send("TEARDOWN")
a.read_response()
a.close()
b.send("TEARDOWN")
b.read_response()
b.close()
say(
case,
fa > 10 and fb > 10 and ga == 0 and gb == 0 and "rejected" in third,
f"two clients: {fa} and {fb} frames in 4 s each, gaps {ga}/{gb}; {third}",
)
elif case == "stall":
a = Client()
setup_play(a)
b = Client()
setup_play(b)
# stop reading on a for 4 s while b keeps playing
a.s.settimeout(None)
fb1, _, gb1, _ = b.read_frames(4)
a.s.settimeout(1.0)
fa, _, ga, _ = a.read_frames(4)
fb2, _, gb2, _ = b.read_frames(3)
a.send("TEARDOWN")
a.read_response()
a.close()
b.send("TEARDOWN")
b.read_response()
b.close()
say(
case,
fb1 > 10 and fb2 > 10 and fa > 5,
f"b kept {fb1} then {fb2} frames while a stalled 4 s; a recovered {fa} frames after resuming (gaps {ga}); b gaps {gb1}/{gb2}; device log must show stalled/dropping frame",
)
elif case == "timeout":
c = Client(timeout=70)
setup_play(c)
c.s.settimeout(
2
) # a server that stops sending but never closes must not hang the run
t0 = time.monotonic()
dead = False
try:
while time.monotonic() - t0 < 68:
try:
d = c.s.recv(65536)
except TimeoutError:
continue
if not d:
dead = True
break
except ConnectionError:
dead = True
elapsed = time.monotonic() - t0
say(
case,
dead and 50 <= elapsed <= 68,
f"connection closed by server after idle: {dead} at {elapsed:.0f} s (expect the 60 s session timeout when the client sends nothing)",
)
except Exception as e: # noqa: BLE001
say(case, False, f"exception {type(e).__name__}: {e}")
time.sleep(1)
print(
"SUMMARY",
sum(1 for r in results if r[1]),
"pass",
sum(1 for r in results if not r[1]),
"fail",
)
sys.exit(1 if any(not r[1] for r in results) else 0)
@@ -0,0 +1,13 @@
#pragma once
#include <zephyr/device.h>
#include <zephyr/drivers/uart.h>
#include "esphome/core/log.h"
// Direct look at the Zephyr cdc_acm_uart1 driver state, bypassing the usb_cdc_acm component
inline void cdc_probe() {
const device *d = DEVICE_DT_GET(DT_NODELABEL(cdc_acm_uart1));
uint8_t tmp[32];
int rr = uart_irq_rx_ready(d);
int n = uart_fifo_read(d, tmp, sizeof(tmp));
ESP_LOGI("probe", "driver rx_ready=%d fifo_read=%d first=0x%02X", rr, n, n > 0 ? tmp[0] : 0);
}
@@ -0,0 +1,203 @@
"""Drive the usb_cdc_acm echo firmware on the nRF52840 dongle from the Mac.
usage: drive.py DATA_PORT [case ...]
Cases: short, bulk1k, bulk16k, bulk64k, burst, flush, peek, linecoding, dtr, stream, counters
"""
import random
import sys
import time
import serial
port = sys.argv[1]
cases = sys.argv[2:] or [
"short",
"bulk1k",
"bulk16k",
"bulk64k",
"burst",
"flush",
"peek",
"linecoding",
"dtr",
"stream",
"counters",
]
results = []
def say(case, ok, detail):
results.append((case, ok, detail))
print(f"{'PASS' if ok else 'FAIL'} {case}: {detail}", flush=True)
def info(case, detail):
"""A case whose result lives in the device log; printed, not counted."""
print(f"INFO {case}: {detail}", flush=True)
def open_port(baud=115200):
s = serial.Serial(port, baudrate=baud, timeout=1)
time.sleep(0.3) # let the 100 ms line-state poll see DTR and the echo loop settle
return s
def read_exact(s, n, timeout):
buf = bytearray()
deadline = time.monotonic() + timeout
while len(buf) < n and time.monotonic() < deadline:
chunk = s.read(min(4096, n - len(buf)))
if chunk:
buf += chunk
return bytes(buf)
def payload(n):
# No 0x01 anywhere: the device treats a chunk starting with 0x01 as a command
rnd = random.Random(n)
return bytes(rnd.choice(range(2, 256)) for _ in range(n))
def cmd(s, c, extra=b""):
s.write(b"\x01" + c + extra)
s.flush()
def echo_case(name, n, chunk):
s = open_port()
s.reset_input_buffer()
data = payload(n)
t0 = time.monotonic()
for i in range(0, n, chunk):
s.write(data[i : i + chunk])
got = read_exact(s, n, timeout=max(5, n / 2000))
dt = time.monotonic() - t0
s.close()
ok = got == data
first_bad = next((i for i in range(min(len(got), n)) if got[i] != data[i]), None)
say(
name,
ok,
f"{len(got)}/{n} bytes echoed in {dt:.2f}s ({n / dt / 1024:.1f} KiB/s), first mismatch {first_bad}",
)
for case in cases:
if case == "short":
s = open_port()
s.reset_input_buffer()
ok = True
lat = []
for i in range(20):
msg = f"hello {i}\n".encode()
t0 = time.monotonic()
s.write(msg)
got = read_exact(s, len(msg), 2)
lat.append(time.monotonic() - t0)
ok = ok and got == msg
s.close()
say(
case,
ok,
f"20 lines, latency min {min(lat) * 1000:.0f} ms max {max(lat) * 1000:.0f} ms",
)
elif case == "bulk1k":
echo_case(case, 1024, 1024)
elif case == "bulk16k":
echo_case(case, 16 * 1024, 4096)
elif case == "bulk64k":
echo_case(case, 64 * 1024, 16384)
elif case == "burst":
s = open_port()
s.reset_input_buffer()
cmd(s, b"B")
got = read_exact(s, 1000, 3)
s.close()
expect = bytes((ord("A") + (i % 26)) for i in range(1000))
prefix_ok = got == expect[: len(got)]
say(
case,
prefix_ok and 0 < len(got) < 1000,
f"{len(got)} of 1000 bytes arrived (256 byte TX ring, rest must be dropped and warned), prefix intact {prefix_ok}",
)
elif case == "flush":
s = open_port()
s.reset_input_buffer()
cmd(s, b"F")
got = read_exact(s, 200, 3)
s.close()
expect = bytes((ord("a") + (i % 26)) for i in range(200))
say(case, got == expect, f"{len(got)}/200 bytes, flush result in device log")
time.sleep(0.5)
elif case == "peek":
s = open_port()
s.reset_input_buffer()
cmd(s, b"K", b"Z")
time.sleep(0.5)
s.close()
info(
case,
"peek/read result is in the device log: expect `peek ok=1 0x5A read ok=1 0x5A`",
)
elif case == "linecoding":
for baud in (9600, 115200, 921600):
s = open_port(baud)
s.reset_input_buffer()
cmd(s, b"L")
time.sleep(0.4)
s.close()
time.sleep(0.3)
info(case, "opened at 9600, 115200, 921600; the device log must show each baud")
elif case == "dtr":
for _i in range(5):
s = open_port()
time.sleep(0.3)
s.close()
time.sleep(0.3)
info(
case, "5 open/close cycles; the device log must show 5 Opened and 5 Closed"
)
elif case == "stream":
# device streams 100 x 100 byte lines over 10 s; close the port for 3 s in the middle
s = open_port()
s.reset_input_buffer()
cmd(s, b"S")
part1 = bytearray()
t0 = time.monotonic()
while time.monotonic() - t0 < 2.5:
part1 += s.read(4096)
s.close()
time.sleep(3.0)
s = open_port()
part2 = bytearray()
t0 = time.monotonic()
while time.monotonic() - t0 < 6.0:
part2 += s.read(4096)
s.close()
lines1 = [x for x in bytes(part1).split(b"\n") if x.startswith(b"STREAM")]
lines2 = [x for x in bytes(part2).split(b"\n") if x.startswith(b"STREAM")]
n1 = [int(x[7:10]) for x in lines1]
n2 = [int(x[7:10]) for x in lines2]
resumed = bool(n2) and max(n2) >= 90
say(
case,
resumed,
f"before close: lines {min(n1) if n1 else None}-{max(n1) if n1 else None} ({len(n1)}), after reopen: {min(n2) if n2 else None}-{max(n2) if n2 else None} ({len(n2)}); streaming resumed after reopen {resumed}",
)
elif case == "counters":
s = open_port()
cmd(s, b"C")
time.sleep(0.3)
s.close()
info(case, "counters are in the device log")
time.sleep(0.5)
print(
"SUMMARY",
sum(1 for r in results if r[1]),
"pass",
sum(1 for r in results if not r[1]),
"fail",
)
sys.exit(1 if any(not r[1] for r in results) else 0)
@@ -0,0 +1,39 @@
#!/bin/zsh
# usage: decode.sh CAPTURE ELF OUTPREFIX
# Parses the #CD: lines of a serial capture with Zephyr's coredump tools from the sdk-nrf that ESPHome
# installed (SDK_NRF, default: the sdk-nrf directory under the ESPHome cache), serves the dump to GDB on a
# random port and prints the backtrace and the thread list. The ELF is .pioenvs/<name>/zephyr/zephyr/zephyr.elf.
# Every step is checked: a missing tool, a capture without a complete dump, an empty .bin or a gdbserver
# that died all stop the script with exit 1, and the exit status is GDB's otherwise.
CAP=$1; ELF=$2; OUT=$3
SDK_NRF=${SDK_NRF:-$(ls -d ~/Library/Caches/esphome/sdk-nrf ~/.cache/esphome/sdk-nrf 2>/dev/null | head -1)}
[ -n "$SDK_NRF" ] || { echo "sdk-nrf not found, set SDK_NRF"; exit 1; }
F=$(ls -d $SDK_NRF/frameworks/v*/zephyr/scripts/coredump 2>/dev/null | tail -1)
PY=${PYTHON:-$(ls $SDK_NRF/penvs/*/bin/python 2>/dev/null | head -1)}
# SDK 0.x keeps the tools under arm-zephyr-eabi/, SDK 1.0+ under gnu/arm-zephyr-eabi/
GDB=$(ls $SDK_NRF/toolchains/*/arm-zephyr-eabi/bin/arm-zephyr-eabi-gdb $SDK_NRF/toolchains/*/gnu/arm-zephyr-eabi/bin/arm-zephyr-eabi-gdb 2>/dev/null | tail -1)
for v in F PY GDB; do
[ -n "${(P)v}" ] || { echo "$v not found under $SDK_NRF"; exit 1; }
done
[ -f "$ELF" ] || { echo "ELF not found: $ELF"; exit 1; }
# strip capture timestamps and ANSI so the parser sees the raw log lines
sed -E 's/^\[[0-9:.]+\]//; s/\x1b\[[0-9;]*m//g' "$CAP" > $OUT-all.log
# keep only the first complete #CD:BEGIN# .. #CD:END# block
awk '/#CD:BEGIN#/{if(!done){p=1;buf=""}} p{buf=buf $0 "\n"} /#CD:END#/{if(p){printf "%s", buf; done=1; p=0}}' $OUT-all.log > $OUT-raw.log
echo "CD lines in capture: $(grep -c '#CD:' $OUT-all.log) (BEGIN $(grep -c '#CD:BEGIN#' $OUT-all.log), END $(grep -c '#CD:END#' $OUT-all.log)); first complete block: $(grep -c '#CD:' $OUT-raw.log) lines"
[ -s $OUT-raw.log ] || { echo "no complete #CD:BEGIN# .. #CD:END# block in the capture"; exit 1; }
if ! $PY $F/coredump_serial_log_parser.py $OUT-raw.log $OUT.bin > $OUT-parser.log 2>&1; then
echo "parser failed:"; tail -5 $OUT-parser.log; exit 1
fi
tail -3 $OUT-parser.log
[ -s $OUT.bin ] || { echo "parser wrote an empty $OUT.bin"; exit 1; }
ls -la $OUT.bin | awk '{print "bin bytes", $5}'
PORT=$((20000 + RANDOM % 10000))
$PY $F/coredump_gdbserver.py --port $PORT "$ELF" $OUT.bin > $OUT-gdbserver.log 2>&1 &
G=$!
sleep 3
if ! kill -0 $G 2>/dev/null; then echo "gdbserver exited:"; tail -5 $OUT-gdbserver.log; exit 1; fi
$GDB -q -batch -ex "target remote :$PORT" -ex "bt" -ex "info threads" "$ELF" 2>&1 | grep -v "^warning\|Reading symbols\|^$" | head -40
rc=$pipestatus[1]
kill $G 2>/dev/null
exit $rc
@@ -0,0 +1,29 @@
#!/bin/zsh
# usage: flow.sh LABEL PORT ELF [OUTDIR]
# Run at least 60 s after the board rebooted from its crash, with the port closed. Proves the stored
# coredump waits for a reader and survives a reader that goes away:
# 1) open the port for 3 s and close it (a reader that leaves mid-print)
# 2) open it for 100 s: the whole dump must print again, then be erased
# 3) open it again for 20 s: dump_config must report no stored dump
# then decode.sh parses the first complete dump and prints the GDB backtrace.
# Needs a python with pyserial (PYTHON, default python3).
D=${0:A:h}
L=$1; P=$2; ELF=$3; S=${4:-.}
PY=${PYTHON:-python3}
CAP=$D/../serial_capture.py
capture() { # SECONDS OUTFILE: stop the flow when the capture fails (busy or missing port)
if ! $PY $CAP $P $1 $2 > $2.err 2>&1; then
echo "capture of $2 failed:"; cat $2.err; exit 1
fi
}
capture 3 $S/$L-brief.txt
echo "$L brief 3 s open: $(wc -l < $S/$L-brief.txt) lines, CD lines $(grep -a -c '#CD:' $S/$L-brief.txt), END $(grep -a -c '#CD:END#' $S/$L-brief.txt)"
sleep 5
capture 100 $S/$L-full.txt
echo "$L full 100 s open: $(wc -l < $S/$L-full.txt) lines, CD lines $(grep -a -c '#CD:' $S/$L-full.txt), BEGIN $(grep -a -c '#CD:BEGIN#' $S/$L-full.txt), END $(grep -a -c '#CD:END#' $S/$L-full.txt)"
grep -a "crashtest\|zephyr_coredump\|Reset reason\|Has stored\| Size:\| Error:\|took a long\|compiled on" $S/$L-full.txt | grep -a -v "#CD:[0-9A-F]" | head -16
sleep 3
capture 20 $S/$L-after.txt
echo "$L reopen 20 s: CD lines $(grep -a -c '#CD:' $S/$L-after.txt); $(grep -a 'Has stored\| Size:' $S/$L-after.txt | sed -E 's/^\[[^]]*\]//' | tr '\n' ' ')"
echo "== decode"
$D/decode.sh $S/$L-full.txt $ELF $S/$L
@@ -0,0 +1,24 @@
import asyncio
import sys
from aioesphomeapi import APIClient
async def main():
c = APIClient(sys.argv[1], 6053, None)
await c.connect(login=True)
di = await c.device_info()
print(
"version",
di.esphome_version,
"compiled",
di.compilation_time,
"home_id",
hex(di.zwave_home_id),
"flags",
di.zwave_proxy_feature_flags,
)
await c.disconnect()
asyncio.run(main())
@@ -0,0 +1,45 @@
"""Recompute the Z-Wave checksum (XOR of LENGTH..payload, initial 0xFF) over every frame the proxy logged."""
from functools import reduce
import pathlib
import re
import sys
def zw_checksum(frame: bytes) -> int:
return reduce(lambda a, b: a ^ b, frame[1:-1], 0xFF)
def complete(frame: bytes) -> bool:
"""A data frame is SOF, LENGTH, then LENGTH bytes; the proxy logs at most 168 bytes, so longer ones are cut."""
return len(frame) >= 3 and frame[0] == 0x01 and len(frame) == frame[1] + 2
for path in sys.argv[1:]:
text = pathlib.Path(path).read_text(encoding="utf-8", errors="replace")
sent = re.findall(r"Sending: ((?:[0-9A-F]{2}:)+[0-9A-F]{2})", text)
recv = re.findall(r"Received frame: ((?:[0-9A-F]{2}:)+[0-9A-F]{2})", text)
calc = re.findall(
r"CHECKSUM Received: 0x([0-9A-F]{2}) - Calculated: 0x([0-9A-F]{2})", text
)
bad = len(re.findall(r"Bad checksum", text))
acks_sent = len(re.findall(r"Sending ACK", text))
naks_sent = len(re.findall(r"Sending NAK/CAN", text))
acks_recv = len(re.findall(r"Received ACK", text))
naks_recv = len(re.findall(r"Received NAK", text))
reboots = len(re.findall(r"Boot count|Reset reason|rst:0x", text))
long_ops = len(re.findall(r"zwave_proxy took a long time", text))
rx_frames = [bytes.fromhex(f.replace(":", "")) for f in recv]
tx_frames = [bytes.fromhex(f.replace(":", "")) for f in sent]
rx_cut = sum(1 for b in rx_frames if not complete(b))
tx_cut = sum(1 for b in tx_frames if len(b) > 1 and not complete(b))
rx_ok = sum(1 for b in rx_frames if complete(b) and zw_checksum(b) == b[-1])
tx_ok = sum(1 for b in tx_frames if complete(b) and zw_checksum(b) == b[-1])
calc_ok = sum(1 for r, c in calc if r == c)
cmds = sorted({f.split(":")[3] for f in recv})
print(
f"{path.rsplit('/', 1)[-1]}: tx frames {len(sent)} (checksum ok {tx_ok}, truncated in log {tx_cut}), "
f"rx frames {len(recv)} (recomputed ok {rx_ok}, truncated in log {rx_cut}), "
f"device calc==received {calc_ok}/{len(calc)}, bad checksum warnings {bad}, ACK sent {acks_sent}, NAK sent {naks_sent}, "
f"ACK received {acks_recv}, NAK received {naks_recv}, long ops {long_ops}, rx cmd ids {cmds}"
)
@@ -0,0 +1,44 @@
substitutions:
run: "1"
esphome:
name: bk72xx-crash-probe
includes: [../scripts/libretiny_crash/bkprobe.hpp]
# step 0 waits 30 s so a capture can be opened (and a reset pulsed) after the flash; later steps 5 s
on_boot:
priority: -100
then:
- lambda: id(step) = bkprobe_boot(${run});
- delay: !lambda 'return id(step) == 0 ? 30000 : 5000;'
- lambda: bkprobe_act(${run}, id(step));
bk72xx:
board: generic-bk7238-tuya
logger:
hardware_uart: UART1
level: DEBUG
preferences:
flash_write_interval: 1s
globals:
- id: step
type: int
initial_value: "0"
interval:
- interval: 10s
then:
- lambda: bkprobe_alive(id(step));
# Wi-Fi matters: a crashed BK72xx with Wi-Fi up never reboots on its own without the crash handler fix
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
api:
reboot_timeout: 0s
ota:
- platform: esphome
@@ -0,0 +1,38 @@
substitutions:
run: "1"
esphome:
name: lt21007e
includes: [../scripts/libretiny_prefs/ltprefs.hpp]
platformio_options:
build_flags: ["-DUSE_LIBRETINY_RTC_PREFERENCES"]
on_boot:
priority: -100
then:
- lambda: id(act) = ltprefs_boot(id(step), ${run});
# the first boot waits so the host can open the port and pulse a reset before the sequence starts
- delay: !lambda 'return (id(step) == 0 || id(step) == 8) ? 30000 : 3000;'
- lambda: ltprefs_exit(id(step), ${run}, id(act));
bk72xx:
board: generic-bk7238-tuya
logger:
hardware_uart: UART1
level: DEBUG
preferences:
flash_write_interval: 1s
globals:
- id: step
type: int
initial_value: "0"
- id: act
type: int
initial_value: "0"
interval:
- interval: 10s
then:
- lambda: ltprefs_alive(id(step), id(act));
@@ -0,0 +1,34 @@
esphome:
name: i19430-dut3
on_boot:
priority: 800
then:
- lambda: |-
auto marker = global_preferences->make_preference<uint32_t>(0x19430001, true);
uint32_t v = 0;
if (!marker.load(&v)) {
ESP_LOGW("test", "first boot: wiping preferences");
global_preferences->reset();
static_cast<esphome::libretiny::LibreTinyPreferences *>(global_preferences)->open();
v = 1;
marker.save(&v);
global_preferences->sync();
}
bk72xx:
board: generic-bk7238
logger:
hardware_uart: UART1
level: VERBOSE
wifi:
ap:
ssid: i19430-ap
captive_portal:
interval:
- interval: 5s
then:
- lambda: ESP_LOGI("test", "alive %u s", (unsigned) (millis() / 1000));
@@ -0,0 +1,50 @@
substitutions:
run: "3"
esphome:
name: cd13122d
on_boot:
priority: -100
then:
- lambda: |-
ESP_LOGI("crashtest", "boot, crashed_before=%d run=%s", id(crashed_${run}), "${run}");
- delay: 20s
- if:
condition:
lambda: 'return id(crashed_${run}) == 0;'
then:
- lambda: |-
id(crashed_${run}) = 1;
ESP_LOGW("crashtest", "flag set, letting the loop save it");
- delay: 3s
- lambda: |-
global_preferences->sync();
delay(500);
ESP_LOGW("crashtest", "writing to 0xDEADBEEF now");
delay(100);
*reinterpret_cast<volatile uint32_t *>(0xDEADBEEF) = 1;
else:
- lambda: |-
ESP_LOGI("crashtest", "already crashed once, running normally");
nrf52:
board: nrf52840dongle
dfu: true
logger:
hardware_uart: USB_CDC
level: DEBUG
zephyr_coredump:
globals:
- id: crashed_${run}
type: int
restore_value: true
initial_value: "0"
interval:
- interval: 10s
then:
- lambda: |-
ESP_LOGI("crashtest", "alive");
@@ -0,0 +1,31 @@
esphome:
name: pref20810
on_boot:
priority: -100
then:
- lambda: |-
ESP_LOGI("test", "boot: restored counter_pr=%d", id(counter_pr));
nrf52:
board: nrf52840dongle
dfu: true
logger:
hardware_uart: USB_CDC
level: VERBOSE
preferences:
flash_write_interval: 5s
globals:
- id: counter_pr
type: int
restore_value: true
initial_value: "0"
interval:
- interval: 15s
then:
- lambda: |-
id(counter_pr) += 1;
ESP_LOGI("test", "counter_pr now %d", id(counter_pr));
@@ -0,0 +1,84 @@
esphome:
name: online-image-dut
friendly_name: online-image-dut
esp32:
board: esp32dev
framework:
type: esp-idf
logger:
level: DEBUG
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
power_save_mode: none
api:
encryption:
key: !secret api_key
reboot_timeout: 0s
actions:
- action: fetch
variables:
url: string
then:
- online_image.set_url:
id: img
url: !lambda 'return url;'
update: false
- component.update: img
- action: refetch
then:
- component.update: img
- action: fetch_small
variables:
url: string
then:
- online_image.set_url:
id: img_small
url: !lambda 'return url;'
update: false
- component.update: img_small
- action: refetch_small
then:
- component.update: img_small
http_request:
timeout: 4.5s
verify_ssl: false
display:
image:
- platform: online_image
id: img
url: http://127.0.0.1/none.bmp
format: AUTO
type: RGB565
update_interval: never
on_download_finished:
logger.log:
format: "download finished: img cached=%d"
args: [cached]
on_error:
logger.log: "download error: img"
- platform: online_image
id: img_small
url: http://127.0.0.1/none.bmp
format: AUTO
type: RGB565
buffer_size: 256
update_interval: never
on_download_finished:
logger.log:
format: "download finished: img_small cached=%d"
args: [cached]
on_error:
logger.log: "download error: img_small"
button:
- platform: restart
id: reboot_btn
name: Reboot
@@ -0,0 +1,105 @@
esphome:
name: rtsp17990
on_boot:
# GPIO8 low enables the camera's power supply on the Waveshare ESP32-S3-ETH; give the sensor
# time to come up before esp32_camera (setup_priority 200) probes it
priority: 700
then:
- switch.turn_off: cam_power
- lambda: |-
delay(1200);
esp32:
board: esp32-s3-devkitc-1
variant: esp32s3
flash_size: 16MB
framework:
type: esp-idf
psram:
mode: octal
speed: 80MHz
logger:
level: VERBOSE
hardware_uart: USB_SERIAL_JTAG
logs:
rtsp_server: VERBOSE
esp32_camera: DEBUG
camera: DEBUG
wifi: DEBUG
api: DEBUG
api.connection: DEBUG
api.socket: DEBUG
socket: DEBUG
scheduler: DEBUG
component: DEBUG
app: DEBUG
mdns: DEBUG
ota: DEBUG
logger: DEBUG
psram: DEBUG
esp32: DEBUG
network: DEBUG
i2c: DEBUG
i2c.idf: DEBUG
preferences: DEBUG
debug: DEBUG
sensor: DEBUG
wifi_esp32: DEBUG
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
power_save_mode: none
api:
reboot_timeout: 0s
ota:
- platform: esphome
debug:
update_interval: 10s
sensor:
- platform: debug
free:
name: "Heap Free"
block:
name: "Heap Largest Block"
psram:
name: "PSRAM Free"
# Waveshare ESP32-S3-ETH camera header (Waveshare wiki, WIFI_Web_CAM table)
i2c:
- id: cam_i2c
sda: GPIO48
scl: GPIO47
frequency: 400kHz
switch:
- platform: gpio
id: cam_power
name: Camera Power
pin: GPIO8
restore_mode: ALWAYS_OFF
setup_priority: 800
esp32_camera:
name: cam
setup_priority: 200
data_pins: [GPIO41, GPIO45, GPIO46, GPIO42, GPIO40, GPIO38, GPIO15, GPIO18]
vsync_pin: GPIO1
href_pin: GPIO2
pixel_clock_pin: GPIO39
external_clock:
pin: GPIO3
frequency: 20MHz
i2c_id: cam_i2c
resolution: 640X480
max_framerate: 10fps
rtsp_server:
port: 554
max_connections: 2
@@ -0,0 +1,107 @@
# Basic Config
esphome:
name: s31-cse7766
esp8266:
board: esp01_1m
toolchain: arduino
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
logger:
baud_rate: 0 # (UART logging interferes with cse7766)
level: VERBOSE
logs:
cse7766: VERBOSE
api: DEBUG
api.connection: DEBUG
api.socket: DEBUG
socket: DEBUG
scheduler: DEBUG
component: DEBUG
app: DEBUG
wifi: DEBUG
mdns: DEBUG
ota: DEBUG
logger: DEBUG
sensor: DEBUG
switch: DEBUG
binary_sensor: DEBUG
status_led: DEBUG
uart: DEBUG
gpio: DEBUG
preferences: DEBUG
esp8266: DEBUG
wifi_esp8266: DEBUG
api:
ota:
platform: esphome
# Device Specific Config
uart:
rx_pin: RX
baud_rate: 4800
parity: EVEN
binary_sensor:
- platform: gpio
pin:
number: GPIO0
mode: INPUT_PULLUP
inverted: True
name: "S31 Button"
on_press:
- switch.toggle: relay
- platform: status
name: "S31 Status"
sensor:
- platform: wifi_signal
name: "S31 WiFi Signal"
update_interval: 60s
- platform: cse7766
current:
accuracy_decimals: 3
name: "S31 Current"
filters:
- throttle: 2s
voltage:
accuracy_decimals: 1
name: "S31 Voltage"
filters:
- throttle: 2s
on_raw_value:
then:
- lambda: |-
id(frames) += 1;
power:
accuracy_decimals: 1
name: "S31 Power"
filters:
- throttle: 2s
switch:
- platform: gpio
name: "S31 Relay"
pin: GPIO12
id: relay
restore_mode: RESTORE_DEFAULT_ON
status_led:
pin: GPIO13
globals:
- id: frames
type: uint32_t
initial_value: "0"
interval:
- interval: 10s
then:
- lambda: |-
static uint32_t last = 0;
ESP_LOGI("frames", "cse7766 frames parsed: %u in 10 s (total %u)", (unsigned) (id(frames) - last), (unsigned) id(frames));
last = id(frames);
@@ -0,0 +1,112 @@
esphome:
name: cdc11634
nrf52:
board: nrf52840dongle
dfu: true
logger:
hardware_uart: USB_CDC
level: VERBOSE
logs:
usb_cdc_acm: VERBOSE
scheduler: DEBUG
component: DEBUG
app: DEBUG
preferences: DEBUG
logger: DEBUG
nrf52: DEBUG
dfu: DEBUG
usb_cdc_acm:
interfaces:
- disabled: true
- id: cdc1
globals:
- id: rx_total
type: uint32_t
initial_value: "0"
- id: tx_attempted
type: uint32_t
initial_value: "0"
- id: stream_left
type: int32_t
initial_value: "0"
interval:
- interval: 20ms
then:
- lambda: |-
uint8_t buf[64];
while (id(cdc1).available() > 0) {
// a command is exactly two bytes, so what the host sends after it stays in the driver for peek/read
uint8_t first = 0;
id(cdc1).peek_byte(&first);
size_t n = std::min<size_t>(id(cdc1).available(), sizeof(buf));
if (first == 0x01 && n >= 2)
n = 2;
if (!id(cdc1).read_array(buf, n)) {
ESP_LOGE("test", "read_array(%u) failed", (unsigned) n);
break;
}
id(rx_total) += n;
if (buf[0] == 0x01 && n >= 2) {
const char cmd = buf[1];
if (cmd == 'B') {
// one oversized write: the 256 byte TX ring must drop the rest and warn once
static uint8_t big[1000];
for (size_t i = 0; i < sizeof(big); i++) big[i] = 'A' + (i % 26);
id(cdc1).write_array(big, sizeof(big));
id(tx_attempted) += sizeof(big);
ESP_LOGI("test", "burst 1000 written");
} else if (cmd == 'S') {
id(stream_left) = 100;
ESP_LOGI("test", "stream started");
} else if (cmd == 'L') {
ESP_LOGI("test", "line coding baud=%u data=%u stop=%u parity=%u", (unsigned) id(cdc1).get_baud_rate(),
id(cdc1).get_data_bits(), id(cdc1).get_stop_bits(), (unsigned) id(cdc1).get_parity());
} else if (cmd == 'K') {
// the host sends "\x01KZ" in one write; the Z must still be in the driver here
uint8_t pb = 0, rb = 0;
bool okp = id(cdc1).peek_byte(&pb);
bool okr = id(cdc1).read_byte(&rb);
ESP_LOGI("test", "peek ok=%d 0x%02X read ok=%d 0x%02X (expect ok=1 0x5A for both)", okp, pb, okr, rb);
if (okr) id(rx_total) += 1;
} else if (cmd == 'F') {
static uint8_t pat[200];
for (size_t i = 0; i < sizeof(pat); i++) pat[i] = 'a' + (i % 26);
id(cdc1).write_array(pat, sizeof(pat));
id(tx_attempted) += sizeof(pat);
const uint32_t t0 = millis();
auto res = id(cdc1).flush();
ESP_LOGI("test", "flush result=%d took %u ms", (int) res, (unsigned) (millis() - t0));
} else if (cmd == 'C') {
ESP_LOGI("test", "counters rx=%u tx_attempted=%u (write_array drops silently when the ring is full)", (unsigned) id(rx_total), (unsigned) id(tx_attempted));
} else {
ESP_LOGW("test", "unknown cmd 0x%02X", cmd);
}
continue;
}
id(cdc1).write_array(buf, n);
id(tx_attempted) += n;
}
- interval: 100ms
then:
- lambda: |-
if (id(stream_left) > 0) {
char line[101];
int len = snprintf(line, sizeof(line), "STREAM %03d ", 100 - id(stream_left));
for (int i = len; i < 99; i++) line[i] = 'x';
line[99] = '\n';
id(cdc1).write_array(reinterpret_cast<const uint8_t *>(line), 100);
id(tx_attempted) += 100;
id(stream_left) -= 1;
if (id(stream_left) == 0) {
ESP_LOGI("test", "stream done");
}
}
- interval: 10s
then:
- lambda: |-
ESP_LOGI("test", "alive rx=%u tx_attempted=%u", (unsigned) id(rx_total), (unsigned) id(tx_attempted));
@@ -0,0 +1,83 @@
esphome:
name: camdiag
on_boot:
priority: -100
then:
- output.set_level:
id: xclk
level: 50%
- delay: 200ms
- lambda: |-
auto *bus = id(cam_i2c);
auto scan = [&](const char *what) {
int found = 0;
for (uint8_t a = 0x08; a < 0x78; a++) {
if (bus->write(a, nullptr, 0) == i2c::ERROR_OK) {
ESP_LOGI("camdiag", " ACK at 0x%02X", a);
found++;
}
}
ESP_LOGI("camdiag", "scan %s: %d devices", what, found);
};
auto rd16 = [&](uint16_t reg) {
uint8_t r[2] = {(uint8_t) (reg >> 8), (uint8_t) reg};
uint8_t v = 0xEE;
auto err = bus->write_readv(0x3C, r, 2, &v, 1);
ESP_LOGI("camdiag", " 0x3C [0x%04X] err=%d -> 0x%02X", reg, (int) err, v);
return v;
};
auto rd8 = [&](uint8_t addr, uint8_t reg) {
uint8_t v = 0xEE;
auto err = bus->write_readv(addr, &reg, 1, &v, 1);
ESP_LOGI("camdiag", " 0x%02X [0x%02X] (8 bit) err=%d -> 0x%02X", addr, reg, (int) err, v);
};
id(g8).turn_off(); delay(1500);
id(xclk).set_level(0.0f); delay(300);
scan("GPIO8 LOW, XCLK OFF");
id(xclk).set_level(0.5f); delay(300);
scan("GPIO8 LOW, XCLK ON");
ESP_LOGI("camdiag", "OV5640 style 16 bit reads (expect 300A=0x56 300B=0x40, 302A=0xB0..B2):");
rd16(0x300A); rd16(0x300B); rd16(0x302A); rd16(0x3008); rd16(0x3100); rd16(0x3103); rd16(0x4300);
ESP_LOGI("camdiag", "8 bit reads at 0x3C (GC2145 expects F0=0x21 F1=0x45):");
rd8(0x3C, 0xF0); rd8(0x3C, 0xF1); rd8(0x3C, 0x0A); rd8(0x3C, 0x0B);
ESP_LOGI("camdiag", "write 0x3103=0x11 then read back, then soft reset 0x3008=0x82 and re-read IDs:");
uint8_t w[3] = {0x31, 0x03, 0x11};
auto werr = bus->write(0x3C, w, 3);
ESP_LOGI("camdiag", " write err=%d", (int) werr);
rd16(0x3103);
uint8_t rst[3] = {0x30, 0x08, 0x82};
bus->write(0x3C, rst, 3); delay(100);
rd16(0x300A); rd16(0x300B);
id(g8).turn_on(); delay(500); scan("GPIO8 HIGH (camera off)");
id(g8).turn_off(); delay(1500); scan("GPIO8 LOW again");
esp32:
board: esp32-s3-devkitc-1
variant: esp32s3
flash_size: 16MB
framework:
type: esp-idf
psram:
mode: octal
speed: 80MHz
logger:
level: DEBUG
hardware_uart: USB_SERIAL_JTAG
i2c:
- id: cam_i2c
sda: GPIO48
scl: GPIO47
frequency: 100kHz
scan: true
output:
- platform: gpio
id: g8
pin: GPIO8
- platform: ledc
pin: GPIO3
id: xclk
frequency: 20000000Hz