Compare commits

..
Author SHA1 Message Date
J. Nick Koston 5849d87a07 Merge remote-tracking branch 'origin/on-boot-direct-callback' into HEAD 2026-11-14 08:40:09 -06:00
J. Nick Koston 113eb659d5 Merge remote-tracking branch 'origin/dev' into HEAD 2026-11-14 08:38:04 -06:00
J. Nick Koston c7b321eba0 Merge remote-tracking branch 'origin/on-boot-direct-callback' into interval-tests 2026-11-11 21:45:19 -06:00
J. Nick Koston a9ae0380cb Build the on_boot callback from codegen expressions instead of f-strings 2026-11-11 21:45:00 -06:00
J. Nick Koston 0bf9e7681a Merge remote-tracking branch 'origin/on-boot-direct-callback' into interval-tests 2026-11-11 12:38:49 -06:00
J. Nick Koston 9d65f4fe20 Merge remote-tracking branch 'origin/dev' into on-boot-direct-callback 2026-11-11 12:38:38 -06:00
J. Nick Koston a48151a993 Merge branch 'on-boot-direct-callback' into interval-tests 2026-11-07 14:13:19 -06:00
J. Nick Koston 3482d33176 [core] Name the DATA priority bit pattern and check it at compile time 2026-11-07 14:13:07 -06:00
J. Nick Koston 6ecd846999 Merge remote-tracking branch 'origin/dev' into on-boot-direct-callback 2026-11-07 14:06:30 -06:00
J. Nick Koston 4912a88528 [core] Restore the checks that each callback runs Fn only from its own method 2026-11-07 14:02:52 -06:00
J. Nick Koston 6eb97bf7ee [interval] Trim the interval codegen and integration tests 2026-11-07 13:48:59 -06:00
J. Nick Koston f41f1c711a Merge branch 'on-boot-direct-callback' into interval-tests 2026-11-07 13:48:38 -06:00
J. Nick Koston 475d794012 [core] Share the callback call target and drop per-callback DATA priority overrides 2026-11-07 13:47:47 -06:00
J. Nick Koston 42ea2a2821 Merge branch 'on-boot-direct-callback' into interval-tests 2026-11-07 13:33:03 -06:00
J. Nick Koston e0e25ac7cc Merge remote-tracking branch 'origin/dev' into on-boot-direct-callback 2026-11-07 13:28:39 -06:00
J. Nick Koston 948a9eeb27 [core] Use bit_cast for the startup callback priority now that it folds 2026-11-04 21:10:47 -06:00
J. Nick Koston 65a83c12b6 Merge remote-tracking branch 'origin/dev' into HEAD 2026-11-04 21:09:03 -06:00
J. Nick Koston 7d8b9c31b2 Merge branch 'on-boot-direct-callback' into interval-tests 2026-11-04 18:35:40 -06:00
J. Nick Koston d9d8518171 Merge remote-tracking branch 'origin/dev' into on-boot-direct-callback
# Conflicts:
#	esphome/cpp_generator.py
#	tests/unit_tests/test_cpp_generator.py
2026-11-04 18:34:31 -06:00
J. Nick Koston e6f6c0f09b Merge on-boot-direct-callback 2026-11-02 09:46:22 -06:00
J. Nick Koston 7815874624 Merge dev 2026-11-02 09:46:07 -06:00
J. Nick Koston d9082378c5 Update the interval codegen tests for intervals with a startup delay 2026-11-02 09:34:59 -06:00
J. Nick Koston 95f7423424 Merge dev 2026-11-02 09:33:12 -06:00
J. Nick Koston fef7a15521 Merge on-boot-direct-callback 2026-11-02 09:33:06 -06:00
J. Nick Koston 0b6fc2c3c5 Merge remote-tracking branch 'upstream/dev' into on-boot-direct-callback
# Conflicts:
#	tests/component_tests/automation/test_core_automations.py
2026-10-31 07:09:54 -10:00
J. Nick Koston 15810c176c Include application.h for HasLoopOverride and check that resume runs the interval again 2026-10-30 21:52:58 -10:00
J. Nick Koston 436baa9375 Merge remote-tracking branch 'origin/on-boot-direct-callback' into interval-tests 2026-10-30 21:50:33 -10:00
J. Nick Koston 4705fbb703 Merge remote-tracking branch 'origin/on-boot-tests' into on-boot-direct-callback
# Conflicts:
#	tests/component_tests/automation/test_core_automations.py
2026-10-30 21:46:06 -10:00
J. Nick Koston ce70abbbb4 Pin that the delayed boot step runs after setup and find automations by trigger 2026-10-30 21:42:15 -10:00
J. Nick Koston 6b5ae000d6 Give the named interval a startup delay 2026-10-30 21:41:24 -10:00
J. Nick Koston 2cfd2a8f51 [interval] Add tests for interval code generation
Pin the IntervalTrigger construction, update interval, startup delay,
registration order and source, setup priority override, and that an
interval runs repeatedly, waits for its startup delay and can be
suspended by id.
2026-10-30 21:35:07 -10:00
J. Nick Koston 77574a50eb Share static_function's declare and define steps and the priority bit cast 2026-10-30 21:34:12 -10:00
J. Nick Koston c52beb6d5e Fold the priority to a constant on every toolchain and share one loop
GCC 10 does not inline the bit_cast fallback at -Os and IDF builds memcpy
as a call, so read the bits with __builtin_memcpy.
2026-10-30 21:25:32 -10:00
J. Nick Koston dde3aaf50e Register the component before its actions are built
Building the actions can wait for ids declared later, so registering after
them moved the block behind other components of the same priority. The
function is now named after the component and declared first; when an
action is not inline it triggers the Automation, so every block without a
hand-set trigger_id uses the small component.
2026-10-30 21:21:50 -10:00
J. Nick Koston 6f7af22b41 Merge branch 'on-boot-tests' into on-boot-direct-callback 2026-10-30 21:20:48 -10:00
J. Nick Koston de0bf34e84 Pin where on_boot registers when its actions wait for other ids 2026-10-30 21:20:45 -10:00
J. Nick Koston 7bd7547983 [core] Run on_boot from a direct setup callback when every action is inline
An on_boot, on_shutdown or on_loop block whose actions are all inline now
builds one small component that calls the generated function from setup(),
on_shutdown() or loop(), instead of a Trigger, an Automation and action
objects. The setup priority is a template argument, so it takes no RAM.
Blocks with a delay, a wait or a trigger_id set by hand keep the Trigger.
2026-10-30 21:16:27 -10:00
J. Nick Koston aae52f2265 [core] Add tests for on_boot code generation and ordering
Pin the code generated today for on_boot, on_shutdown and on_loop
(priorities, registration order, the Trigger kept for a delay or a
trigger_id set by hand) and add an integration test that on_boot blocks
run once in priority order.
2026-10-30 21:05:09 -10:00
730 changed files with 5322 additions and 29208 deletions

No files matched your search

+5 -37
View File
@@ -20,9 +20,7 @@ driven, what the logs show, and the same flow on a `dev` build when a quirk coul
down. Never run `esphome logs` in a `while true` loop: the loop respawns it and holds the port.
2. **Credentials stay local.** Put a `secrets.yaml` next to the scratch configs and reference it with
`!secret`. A copy of a device's own config keeps using the secrets file next to that config: two
checkouts can hold different networks, and the wrong one leaves the device unreachable after the OTA.
Generate an `api_key` with `openssl rand -base64 32` when you have none. Never paste a config with
credentials into a PR, never commit scratch configs or logs.
checkouts can hold different networks, and the wrong one leaves the device unreachable after the OTA. Never paste a config with credentials into a PR, never commit scratch configs or logs.
3. **Build from the tree under test.** `python -m esphome` puts the current directory first on
`sys.path`, so run it with the checkout or worktree as the working directory and pass the yaml by
its absolute path. A venv `esphome` entry point may import an installed snapshot instead of the
@@ -49,15 +47,6 @@ driven, what the logs show, and the same flow on a `dev` build when a quirk coul
piped, or lines land minutes late. Bound every long command with `timeout` (coreutils) or
`perl -e 'alarm shift; exec @ARGV' <seconds> <command>` on macOS without it. Save an exit code with `rc=$?`
right after the command: in `echo "$(date +%T) rc=$?"` the command substitution runs first and `$?` reads 0.
8. **Give builds a private PlatformIO directory when sessions may use different Pythons.** ESPHome stamps the
PlatformIO core directory with the Python version that last built, and a build from another version wipes and
reinstalls the shared packages under every other build. Set `PLATFORMIO_CORE_DIR=<scratch>/pio` on every
compile and upload (`scripts/pio_build.sh` does). Old releases and framework A/B builds need one too
(see `reference/boards.md`).
9. **One ESP-IDF or LibreTiny build at a time on a shared machine.** Concurrent builds race over tool caches. Wrap
compiles in a lock (`BUILD_LOCK` for `scripts/pio_build.sh`, for example a script that holds a `mkdir` lock).
10. **Check that the stimulus ran.** A load or soak run where most requests failed was not loaded, and a capture
that stopped early covers nothing; check the driver's own counters before reading the device's verdict.
## Workflow
@@ -92,17 +81,7 @@ driven, what the logs show, and the same flow on a `dev` build when a quirk coul
> Automated testing results
```
Then one short sentence saying what was tested (builds and SHAs, board, what was driven), and on the next line a
bold result, one of **Test result: PASS**, **Test result: FAIL**, **Test result: PASS WITH CAVEATS** or
**Test result: UNCLEAR**, followed by one sentence with the key numbers (for example "the device closed none of 32
streams, dev closed 12 of 16"). FAIL means a regression or a fix that does not work. PASS WITH CAVEATS means it works
but something about the change itself needs a look; name the caveat in that sentence. A limit of the test (the
original crash did not reproduce) or a device or network quirk the tooling reported correctly is not a caveat on
the change; say it as a separate note. UNCLEAR means the run could not decide; say what is
missing. A reader must not have to dig through the table or guess. Side findings that do not change the result (a
bug already on `dev`, a one-off oddity) go in the bullets, marked as separate from the result.
Then a results table
Then one short sentence saying what was tested (builds and SHAs, board, what was driven), a results table
(build SHA, board and framework, what was driven, counts, result, plus the `dev` control row), short bullets
for the findings, and the setup and exact flow in a collapsed `<details>` block. Answer any question asked on
the PR directly in the comment. No credentials, SSIDs, IP addresses, port names or other bench details.
@@ -139,13 +118,8 @@ ESP-IDF OTA image: `.esphome/build/<name>/build/firmware.ota.bin`. LibreTiny OTA
and then; retry up to three times.
- `api_button.py HOST KEY list|press NAME [...]`: list a device's template buttons or press them over the API; the
recipes drive their test configs with it.
- `apiconn.py`: `connect(host, tries, port, on_stop)` (API connect and login with retries, `API_KEY` from the env),
`log_text()` (an API log line without ANSI codes), `stop_event()` (an Event set when the connection ends),
`ble_connect()` (wait for an advert, connect through a proxy without a cache like Home Assistant) and
`enable_notify()` (start notify and write the CCCD), imported by the API drivers.
- `pio_build.sh compile|upload NAME YAML [DEVICE] [esphome options]`: build or upload YAML as `-s name NAME` from
`TREE` with a private PlatformIO directory, an import check and an optional `BUILD_LOCK`; logs to `NAME-*.log`.
- `resolve.sh NAME [TRIES]`: the address of `NAME.local` once it resolves and answers a ping.
- `apiconn.py`: `connect(host, tries)` (API connect and login with retries, `API_KEY` from the env) and `log_text()`
(an API log line without ANSI codes), imported by the newer API drivers.
- `wifisave_hammer.sh KIND DUT_PORT DUT_YAML DUT_TREE CLIENT_PORT CLIENT_YAML CLIENT_TREE [SETTLE]`:
the fallback AP plus captive portal hammer end to end (compile, flash, capture, counts) with the
templates below. KIND is `esp`, `pico`, `bw15` or `bk`. It ends with `RESULT PASS` (exit 0),
@@ -160,7 +134,7 @@ recorder, dump saver and decoder, cover cycle driver), `online_image/`
(host driver, Zephyr driver probe), `rtsp_server/` (protocol client, raw rate reader, measurement set,
API image grab), `cse7766/` (API observer with ping), `zephyr_coredump/` (reader flow, decode with the
Zephyr tools), `libretiny_prefs/` (rtc preference harness header), `libretiny_crash/` (BK72xx crash probe header,
capture decoder, RTL8720C ROM dump versus report matcher), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `ld2450/` (zone number driver, loopback harness driver with number and binary sensor states and TX frame summarizer, frame stream harness with presence hold and link loss drivers, BT MAC driver), `api/` (raw plaintext native API client for user defined actions, store_yaml recovery client over Noise or plaintext), `http_request/` (update manifest server, update state driver, request test server, action case driver, concurrency, reboot, update check and OTA drivers, `ota_auth/` OTA request cases with basic auth), `modem/` (button and sensor driver over the plaintext API, also used for the BLE MIDI boards, AT commands for the UART passthrough), `dfrobot_sen0395/` (API driver and case runner), `homeassistant_light/` (direct bulb driver, held-report case driver), `usb_audio/` (URL player), `speaker_media_player/` (API player driver, pipeline task case runner), `web_server/` (SSE client, request mix, SSE and HTTPD case runners, per-session `/events` listing check with its config, `/events` head, multi-client, stall, Wi-Fi drop and churn modes with their config, head comparer and runner), `mqtt/` (`ha_discovery/`, `disable/`, `retained_burst/` and `reconnect/` runners; broker config with TLS, client-certificate and WebSocket listeners, test CA with server and client certificates, transport case runner, flash-and-drive runner, CLI TLS check without a board, MAC suffix runner with per-variant configs for two boards, `shutdown/` shutdown message runner and matcher), `udp/` (datagram recorder, `udp.write` payload checker and case driver), `entity_icons/` (icon and device class readback over the API, MQTT discovery, web_server and dump_config, with a comparer), `uart_sim/` (API driver and flash, capture and drive runner for the simulated UART sensors), `remote_base/` (loopback runner and API log decoder per sent frame), `xiaomi_ble/` (frame and header generator with the expected counts, injecting, advertising and receiving configs, counter and runner), `ble_terminate/` (proxy and peer configs, connect and disconnect driver, peer disconnect matcher, runner), `i2c_slave/` (D1 mini slave and BK7238 master configs), `wpa3_ap/` (PMF AP with join and leave logging, BK7238 and CB3S DUTs, runner and summary), `wisp_ap/` (WISP upstream with AP buttons, WISP and ALWAYS_ON DUTs, client, runner and per phase summary), `wifi_restart/` (WPA3 mixed WISP upstream, static IP DUTs with syslog, runner and summary), `ota_timing/` (OTA upload timer with deflate, raw and TCP modes, API build check, A/B runner and median summary, `inflate_bench/` on-device inflate benchmark with stream generator, configs and syslog runner), `gpio_hold/` (ISR pin hold self test with ESP32 and S3 configs), `ble_event_inject/` (GAP and GATTC event injector header and proxy package), `light_sleep_wake/` (LEDC wave and edge timer for light sleep wakeup on a C3, external wave generator and per-call edge detail probe for peripheral power-down on the P4), `manual_ip/` (station and AP manual IP configs, DNS probe and runners), `web_handlers/` (late handler, web_server, captive_portal and prometheus variants, web OTA runner), `bk72xx_flash/` (flash readback with retries, per partition diff of two dumps), `ble_server_security/` (secure GATT server and pairing client configs, driver, summary and runner), `prometheus/` (datetime metrics configs and driver), `zephyr_ble_server/` (nRF52 server, looping ESP32 client and UF2 flash and count runner), `ble_adv_triggers/` (advertiser with six raw frames, ESP32 and BK72xx receivers with eleven triggers, runner and checker), `sun/` (longitude sweep package, runner and checker), `sensor_filters/` (every filter shape with a fixed input sequence, per-board packages and runner), `led_driver_sniff/` (SPI slave capture header with selftest, config generator, window comparer and runner), `hub_input_sim/` (keypad, wiegand and rdm6300 simulation on a D1 mini, API log runner), `value_range/` (on_value_range sequence with a restart, runner), `ezo_fake_bus/` (fake EZO I2C bus and step config, runner), `twai_self/` (TWAI self-reception header, on_frame config and counter runner), `ota_automations/` (OTA automation package for both OTA platforms, four case runner, repeated web upload runner and per case summary), `output_button/` (pad edge timer header, button package and runner), `homeassistant_button/` (raw plaintext API client for action calls and wizard inputs, package and runner), `uart_button/` (UART0 loopback package, API driver with TX and RX counts, runner), `interval/` (plain and delayed interval package, timing summary and runner), `smp_udp_ota/` (SMP reset fault injector, OTA runner, per mode result check, MCUboot swap limit, running image check and slot blanking UF2, Thread device config), `ld2420_sim/` (fake LD2420 answering the hub's commands on a UART0 loopback, API state recorder, two-build comparer and runneLine truncated
capture decoder, RTL8720C ROM dump versus report matcher), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `ld2450/` (zone number driver, loopback harness driver with number and binary sensor states and TX frame summarizer, frame stream harness with presence hold and link loss drivers, BT MAC driver), `api/` (raw plaintext native API client for user defined actions, store_yaml recovery client over Noise or plaintext), `http_request/` (update manifest server, update state driver, request test server, action case driver, concurrency, reboot, update check and OTA drivers, `ota_auth/` OTA request cases with basic auth), `modem/` (button and sensor driver over the plaintext API, also used for the BLE MIDI boards, AT commands for the UART passthrough), `dfrobot_sen0395/` (API driver and case runner), `homeassistant_light/` (direct bulb driver, held-report case driver), `usb_audio/` (URL player), `speaker_media_player/` (API player driver, pipeline task case runner), `web_server/` (SSE client, request mix, SSE and HTTPD case runners), `mqtt/` (`ha_discovery/`, `disable/`, `retained_burst/` and `reconnect/` runners; broker config with TLS, client-certificate and WebSocket listeners, test CA with server and client certificates, transport case runner, flash-and-drive runner, CLI TLS check without a board, MAC suffix runner with per-variant configs for two boards, `shutdown/` shutdown message runner and matcher), `udp/` (datagram recorder, `udp.write` payload checker and case driver), `entity_icons/` (icon and device class readback over the API, MQTT discovery, web_server and dump_config, with a comparer), `uart_sim/` (API driver and flash, capture and drive runner for the simulated UART sensors), `remote_base/` (loopback runner and API log decoder per sent frame), `xiaomi_ble/` (frame and header generator with the expected counts, injecting, advertising and receiving configs, counter and runner), `ble_terminate/` (proxy and peer configs, connect and disconnect driver, peer disconnect matcher, runner), `i2c_slave/` (D1 mini slave and BK7238 master configs), `wpa3_ap/` (PMF AP with join and leave logging, BK7238 and CB3S DUTs, runner and summary), `wisp_ap/` (WISP upstream with AP buttons, WISP and ALWAYS_ON DUTs, client, runner and per phase summary), `wifi_restart/` (WPA3 mixed WISP upstream, static IP DUTs with syslog, runner and summary), `ota_timing/` (OTA upload timer with deflate, raw and TCP modes, API build check, A/B runner and median summary, `inflate_bench/` on-device inflate benchmark with stream generator, configs and syslog runner), `gpio_hold/` (ISR pin hold self test with ESP32 and S3 configs), `ble_event_inject/` (GAP and GATTC event injector header and proxy package), `light_sleep_wake/` (LEDC wave and edge timer for light sleep wakeup on a C3, external wave generator and per-call edge detail probe for peripheral power-down on the P4), `manual_ip/` (station and AP manual IP configs, DNS probe and runners), `web_handlers/` (late handler, web_server, captive_portal and prometheus variants, web OTA runner), `bk72xx_flash/` (flash readback with retries, per partition diff of two dumps), `ble_server_security/` (secure GATT server and pairing client configs, driver, summary and runner), `prometheus/` (datetime metrics configs and driver), `zephyr_ble_server/` (nRF52 server, looping ESP32 client and UF2 flash and count runner), `ble_adv_triggers/` (advertiser with six raw frames, ESP32 and BK72xx receivers with eleven triggers, runner and checker), `sun/` (longitude sweep package, runner and checker), `sensor_filters/` (every filter shape with a fixed input sequence, per-board packages and runner), `led_driver_sniff/` (SPI slave capture header with selftest, config generator, window comparer and runner), `hub_input_sim/` (keypad, wiegand and rdm6300 simulation on a D1 mini, API log runner), `value_range/` (on_value_range sequence with a restart, runner), `ezo_fake_bus/` (fake EZO I2C bus and step config, runner), `twai_self/` (TWAI self-reception header, on_frame config and counter runner), `ota_automations/` (OTA automation package for both OTA platforms, four case runner, repeated web upload runner and per case summary), `output_button/` (pad edge timer header, button package and runner), `homeassistant_button/` (raw plaintext API client for action calls and wizard inputs, package and runner), `uart_button/` (UART0 loopback package, API driver with TX and RX counts, runner), `interval/` (plain and delayed interval package, timing summary and runner), `smp_udp_ota/` (SMP reset fault injector, OTA runner, per mode result check, MCUboot swap limit, running image check and slot blanking UF2, Thread device config), `ld2420_sim/` (fake LD2420 answering the hub's commands on a UART0 loopback, API state recorder, two-build comparer and runner), `nrf52_tcp_probe/` (TCP and UDP socket probe config, build and log check, API check over Thread), `rs485_frame/` (simulated DLE/STX/ETX bus device, frame and Line truncated
## Templates in `$SKILL/templates`
@@ -169,8 +143,6 @@ capture decoder, RTL8720C ROM dump versus report matcher), `api_outgoing/` (wake
- `client-wifisave.yaml` (package): joins that AP, posts N bogus `/wifisave` requests, then the real
credentials from `!secret`. `example-client.yaml` is the per-run file. `bogus_saves: "1"` gives a
plain provisioning check.
- `diagnostics.yaml` (package): DEBUG logs, esphome OTA, uptime, free heap, reset reason and a 200 ms counter
for soak and load tests.
- Per-topic device configs, each described in `reference/recipes/<component>.md`: `online-image-dut.yaml`,
`usb-cdc-acm-dongle-dut.yaml`, `nrf52-preferences-dut.yaml`, `rtsp-waveshare-s3-eth-dut.yaml`,
`waveshare-s3-eth-camdiag.yaml`, `s31-cse7766-dut.yaml`, `nrf52-coredump-crash-dut.yaml`,
@@ -224,10 +196,6 @@ comment quotes the exact commits. Fetch `origin dev` before computing a merge ba
the shared `~/.platformio/platforms/libretiny`, so every later build on the machine uses the patched copy. Patch
through the ESPHome side (a linker `-Wl,-wrap,<symbol>` or a define) instead, and if it happened, restore the
directory from a clean copy of the same release.
- Copying a .uf2 to a mounted bootloader drive (`RPI-RP2`, `ITSY840BOOT`) from a terminal on macOS can fail with
"Operation not permitted" (the Removable Volumes privacy permission), and picotool then sees no device. Flash through
the serial path instead (`esphome upload --device <port>`: the 1200 baud touch on RP2, serial DFU on Adafruit nRF52
boards) or by OTA; a board that only shows the drive needs the file dragged in Finder.
- `wc -l` pads its count on macOS, so `tail -n +$n0` needs `$((n0+0))` or an arithmetic expansion.
- `api:` reboots a board after 15 min without a client: `reboot_timeout: 0s` on `api:` and `mqtt:` for
soak tests.
@@ -29,9 +29,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
interrupt handler; `Software Watchdog` with `pm_send_nullfunc` means the loop was busy too long. Code that takes
interrupt locks needs a load run to show either (`scripts/esp8266_lock_load/`, see recipes/one_wire.md).
- `GPI` bits 16 to 31 are strapping bits (mask with 0xFFFF). GPIO16 has no interrupt; read `GP16O` for an output.
- Cover positions and other restored entity state live in RTC memory by default, so they survive `erase_flash`
and a dev build's state carries into the next build's first boot. Set `esp8266: restore_from_flash: true` on A/B
configs that erase between builds.
## ESP32 classic devkit
@@ -53,8 +50,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
- An ESP32 whose bootloader has no app rollback is protected only by `safe_mode` after a bad OTA. Before sending a
change to such a device that nobody can reach by hand, boot the same code on a devkit first and keep a fallback
image of the merge base built.
- On modules where GPIO16 and GPIO17 are taken by the module itself (PSRAM on WROVER modules, for example), a
`uart:` bus on those pins boot-loops with a watchdog reset during setup. Use GPIO21 and GPIO22.
## ESP32-S3, C3, C6, H2 with native USB
@@ -77,25 +72,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
- A board left in the ROM bootloader after an upload (BOOT held, or the USB JTAG reset landing in download mode
again) starts its app without a replug through `esptool --port <port> --after watchdog-reset chip-id`; flash a
minimal Wi-Fi plus API image first when recovering from a crash-looping one.
- On macOS a software reboot of an ESP32-C3 does not drop its USB JTAG serial port: pyserial keeps reading and gets
no error, so a client that watches for a port reset misses it. Count ESPHome's `Running through setup()` line
instead.
- W5500 SPI Ethernet with IPv6 on (seen on an ESP32-S3, #22676): every `safe_reboot()` (OTA, restart button)
aborts while an API client is subscribed to logs. `mdns_free()` leaves the IPv6 group inside `tcpip_api_call`, the
W5500 driver logs "IPv6 multicast is always filtered in by W5500", the logger sends that line to the API inline,
and the nested `LwIPLock` in `setsockopt(TCP_NODELAY)` releases the core lock it did not take, so `udp_recv`
asserts. No crash with IPv6 off, with no log subscriber (a connected client that does not subscribe is fine), or
on RMII Ethernet. Reading the crash record afterwards: `recipes/crash_report.md`.
- `usb_serial_jtag` holds a `NO_LIGHT_SLEEP` power lock by itself, so a light sleep soak on a native USB board logs
to UART0 instead (see `recipes/light_sleep_gpio_wake.md`).
- The Waveshare ESP32-P4-WIFI6 needs `engineering_sample: true`. Its Wi-Fi is an ESP32-C6 over SDIO through
`esp32_hosted`; a test with no network logs over UART0 to the CH343 console.
## ESP32-C2 (ESP8684, 26 MHz crystal)
- Boards with a 26 MHz crystal print the ROM banner (`ESP-ROM:esp8684`) at 74880 baud; at 115200 it is garbage.
- ESP-IDF builds need `esp32: framework: sdkconfig_options: {CONFIG_XTAL_FREQ_26: y}`, or the app console runs at
the wrong baud rate (the upload log warns `Crystal frequency mismatch`).
## RP2040 / RP2350 (Pico W, Pico 2 W)
@@ -123,25 +99,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
each RD_REQ with one byte from a 256-byte register file, so any read length works (used for BME280/BMP280 in
esphome#22236). The ESP32-S3 I2C slave v2 driver is not usable for register emulation: answering from a task or
straight from the interrupt both gave shifted multi-byte reads.
- RP2350 PIO instruction memory survives a reflash (the 1200 baud touch and the UF2 reboot), so a program loaded by
the previous firmware can hide a missing `pio_add_program`. Clear it from an `on_boot` priority 1000 lambda
(`pio1->instr_mem[i] = 0` for i < 32) to start from power-on state; `scripts/pio_probe/pio-common.yaml` does.
- `RP2Preferences::reset()` erases the flash sector but not the RAM copy the preferences load from, so the LibreTiny
`on_boot` wipe-once pattern still boots with the old saved Wi-Fi credentials. Use a fallback AP config with
`networks:` and a fresh `name:` per build instead.
- WIZnet W5100S-EVB-Pico (RP2040, `rp2: board: wiznet_5100s_evb_pico`): W5100S on SPI with CLK 18, MOSI 19, MISO 16,
CS 17, INT 21, RST 20 (`ethernet: type: W5100`). ESPHome firmware gives it no USB CDC port; reach it over Ethernet,
OTA by address, and flash it over USB only through BOOTSEL and the UF2 drive. Keep `interrupt_pin` in every build:
without it OTA starves and never completes (`recipes/arduino_pico.md`), which strands the board until a USB flash.
The W5100S answers ICMP in hardware, so a ping reply says nothing about the RP2040; probe TCP instead (OTA port
2040, API 6053, HTTP 80). In safe mode only 2040 answers.
- Through a Pico W `bluetooth_proxy`, about 1 to 2 in 5 BLE connects fail with 0x3E (`connect err=62` in the
client) on every build; retry.
- arduino-pico framework bumps and the RPAsyncTCP SSE stall: see `recipes/arduino_pico.md`.
- USB CDC sends nothing until the host asserts DTR; a client that opens with DTR low (`improvcli.py` without
`--keep-dtr`) sees silence.
- Powering a breakout from the Pico: use 3V3(OUT), pin 36. Pin 37 is 3V3_EN, which switches the Pico off when pulled
low.
## BW15 kit (RTL8720CF, LibreTiny rtl87xx)
@@ -183,12 +140,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
- Wrong-MD5 OTA test without changing firmware: monkeypatch `espota2.hashlib.md5` so only the image checksum (data
over 64 KB) returns a wrong hexdigest. Give each build its own `name:` and read `device_info().name` after a
restart to see which image booted.
- Logger UART: `DEFAULT` is UART2 (the kit's USB serial). `hardware_uart: UART1` hangs after the boot banner and
never joins Wi-Fi, on dev too. `UART0` moves both ESPHome and `lt_log` output off the kit's port. LibreTiny's own
`LT_I` lines need `framework: loglevel: info` (the default is warn).
- A config with no Wi-Fi network and no `ap:` resets about 10 s into setup when it has `web_server:` (dev too):
`web_server` setup blocks with no interface up, and no crash record is left. VERBOSE logs show the last
`Setup <component>` line.
## BK7238, WB3S, CB3S (BK7231N) (LibreTiny bk72xx, CH340, RTS on CEN)
@@ -314,10 +265,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
partition even when the upgrade state is still `UPG_STATE_DOWNLOAD_ING` (esphome#22111).
- Without serial access to recover, never send a corrupted image; wrong-MD5 and cut-upload tests with valid
images are safe.
- On the WL2H-U `logger: hardware_uart: DEFAULT` resolves to UART1, LibreTiny's log port for that board.
- Over a poor link (about 60 ms ping and 5 % loss) an ESPHome OTA takes anywhere from 50 to 445 s and a web `/update`
of a UF2 can abort early while ESPHome OTA still works. Treat such a board's results as crash or no crash only, and
settle questions about rates, heap after streams or OTA aborts on a BK7238 (same LibreTiny AsyncTCP).
## nRF52840
@@ -347,10 +294,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
the port open, and the logger warns `logger took a long time`. Print one line per `loop()` instead.
- Zephyr settings survive a reflash: key a `restore_value` global by a `run` substitution when a test needs
a never-saved value.
- `2fe3:0100` is Zephyr's default USB VID:PID, so `ports.py` labels any board running a Zephyr ESPHome build as an
nRF52840 Dongle. Tell an ItsyBitsy from a PCA10059 with a 1200 baud touch: the ItsyBitsy shows its `ITSY840BOOT`
UF2 volume, the PCA10059 its Nordic DFU port.
- nRF52 Zephyr builds use newlib; the nRF54LM20 builds use picolibc.
- Thread: nRF52 `openthread:` accepts only `tlv:`; the individual keys (network_key, channel, ...) are rejected at
validation. To list nearby networks with channel and PAN ID, call
`otThreadDiscover(InstanceLock::acquire().get_instance(), 0, 0xFFFF, false, false, cb, nullptr)` from an
@@ -404,16 +347,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
- BDK headers such as `power_save_pub.h` have no `extern "C"`; wrap them when including them from the Arduino
core's C++ files.
## Shared PlatformIO directory
- The stamp file is `.esphome.pio.stamp.json`; a mismatch deletes `packages`, `platforms` and `.cache`.
- Building an old release (2025.7.0, Arduino ESP32) against the shared directory swapped `tool-scons` for an older
version and broke it midway (`No module named 'SCons.SConf'`) for every PlatformIO build until a newer build
reinstalled it. Build old releases with their own `PLATFORMIO_CORE_DIR`.
- Two arduino-pico framework versions share the plain `framework-arduinopico` package directory, so an A/B swaps the
framework on every build; give each side its own `PLATFORMIO_CORE_DIR`.
- The general rule (a private directory whenever sessions may use different Pythons) is ground rule 8 in `SKILL.md`.
## LibreTiny crash and abort() facts (esphome#21596, #22288)
- `abort()` on dev: the BK7238 hangs for good (the main loop stops and the watchdog never fires once Wi-Fi is up;
@@ -493,33 +426,3 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
- LED0 P1.22, LED1 P1.25; BUTTON0 P1.26, BUTTON1 P1.09, BUTTON2 P1.08, BUTTON3 P0.05 (active low, pull-up). GPIO
P1 registers at 0x500D8200 (IN +0x00C, PIN_CNF +0x080), P0 at 0x5010A200.
- Binary sensor state changes are not logged at DEBUG; log the states or count `on_press` to see button presses.
- FICR INFO.PART reads 0x054BC20B; the build target is `nrf54lm20dk`.
- Zephyr `micros()` on nRF has 30.5 us resolution. Time short blocks with the DWT cycle counter instead; nRF54L idle
clears `DWT_CTRL`, so enable it again before each timed block.
- The nrf52 logger defaults to USB_CDC (the chip's own USB); set `logger: hardware_uart: UART0` to get the log on the
J-Link VCOM port.
- I2C (#22642): TWIM21 only takes pins on P1 or P3; P0 pins are rejected at config. BUTTON1 (P1.09) and BUTTON2
(P1.08) pads work as SDA and SCL. A bus with no pull-ups times out (-116) on every scan address, so setup takes
about 56 s. The JLinkExe `st` command reads VTref to check the I/O voltage.
## Seeed XIAO nRF54LM20A
- CMSIS-DAP probe (`2886:0068`) with one serial port that carries the console (uart20 on P1.11 and P1.10, #22599).
It ships with a Seeed demo.
- Flash with `esphome run` (pyOCD, MCUboot plus the signed app). A plain `esphome upload` flashed only the bare app
on #22599, and the board stayed dark.
- Reset: `<sdk-nrf python> -m pyocd reset -t nrf54lm20a -u <probe id>`.
- nPM1300 PMIC at 0x6B on P1.18 (SDA) and P1.17 (SCL) (#22661; `i2c:` builds on i2c21 at 100 kHz). Per the upstream
Zephyr board files, BUCK2 supplies VSYS_3V3, the nRF54 and the SAMD11 USB debugger: never switch it off (there is no
way back without a power cycle, and the probe dies with it). LDO1 feeds the IMU and microphone (3.3 V, LDO mode) and
is safe to toggle. BUCK1 and LDO2 are not described anywhere; leave them alone.
- nPM1300 power-on state: BUCK2 on at 3.3 V, BUCK1 off, LDO1 and LDO2 off in load switch mode (enabling one as is
passes VINLDO straight through), charger off, VBUS limit 100 mA, NTC select 10 k. Flows: `recipes/npm1300.md`.
## I2C and SPI breakouts
- GPIO switches on an I2C expander (XL9535 and others) write `value_or(false)` at setup even with
`restore_mode: DISABLED`, so test pin retention on a pin no switch uses.
- ADS1015: `sample_rate: 8` maps to 128 SPS at `12_BITS`.
- A bus held low (`Performing bus recovery`, no devices found) on a new breakout can be a bad header joint; reflow
it before blaming the driver.
@@ -4,7 +4,7 @@ Each flow names the pull requests it was used on, so their result tables show wh
One file per component, named after it:
- [`api.md`](api.md): noise handshake and OTA encryption on the ESP8266, outgoing connection wake timing on deep sleep and after a Wi-Fi drop, user defined action listing and execution with a raw plaintext client, store_yaml recovery over Noise or plaintext, rapid connect/disconnect stress (`connstress.py`)
- [`api.md`](api.md): noise handshake and OTA encryption on the ESP8266, outgoing connection wake timing on deep sleep and after a Wi-Fi drop, user defined action listing and execution with a raw plaintext client, store_yaml recovery over Noise or plaintext
- [`ble_midi.md`](ble_midi.md): BLE MIDI client against a fake ESP32 instrument, every packet shape and the send actions
- [`captive_portal.md`](captive_portal.md): fallback AP and /wifisave hammer
- [`cse7766.md`](cse7766.md): Sonoff S31 parser check over the API
@@ -18,12 +18,11 @@ One file per component, named after it:
- [`modem.md`](modem.md): PPPoS modem component boot check with no modem attached, loop blocking and disable timing, SIM7080G behaviour with no SIM and with no data plan through a raw UART passthrough
- [`mqtt.md`](mqtt.md): broker bench, QoS, reconnect, crash log over MQTT, WebSocket transports with a local and a public broker, TLS trust sources (own CA, client certificate, certificate bundle, no verification) with the CLI check, MAC suffix topics and will messages on two boards, shutdown message delivery on restart, disable and deep sleep
- [`nrf52.md`](nrf52.md): dongle board support, zephyr_ble_server connection slot under reordered callbacks, SMP over UDP OTA over Thread with reset faults, MCUboot swap limit and stuck slot recovery, TCP stack probe, sdk-nrf 3.4.1 build and USB notes
- [`npm1300.md`](npm1300.md): nPM1300 PMIC on the XIAO nRF54LM20A: probe build, every entity with a register dump and regulator toggles, power-on state and no-battery readings
- [`online_image.md`](online_image.md): stalled and malformed downloads, PNG, JPEG, BMP and QOI decode time per loop pass with large images and small buffers
- [`openthread.md`](openthread.md): border router on the S3 host plus H2 RCP
- [`ota.md`](ota.md): upload without a configuration and the refusal checks, overlap and lock, timeouts, deflate, upload timing A/B per build and board with deflate and raw, on-device inflate benchmark, OTA automations on two platforms
- [`rtsp_server.md`](rtsp_server.md): camera streaming on the Waveshare ESP32-S3-ETH
- [`scheduler.md`](scheduler.md): loop wake A/B, interval startup delay and control from lambdas, RP2 scheduler lock under IRQ and lwIP load
- [`scheduler.md`](scheduler.md): loop wake A/B, interval startup delay and control from lambdas
- [`udp.md`](udp.md): datagrams, multicast, sockets, `udp.write` payload shapes
- [`usb_cdc_acm.md`](usb_cdc_acm.md): nRF52840 dongle host driver
- [`wifi.md`](wifi.md): reboot_timeout with a fallback AP, DHCP lease cache, Easy Connect (DPP), WISP bridge with an ethernet upstream, reboots against a WPA2/WPA3 PMF access point, WISP and ALWAYS_ON AP switching, AP restarts with WPA3 mixed and LIGHT power save, soft AP clients with the BLE tracker scanning, ESP8266 power save after captive portal provisioning
@@ -37,8 +36,8 @@ One file per component, named after it:
- [`usb_audio.md`](usb_audio.md): USB audio speaker and microphone, the usb_host FIFO split, and Music Assistant playback through sendspin
- [`dfrobot_sen0395.md`](dfrobot_sen0395.md): SEN0395 mmWave command flow, module quirks and overlong lines through its error reply
- [`speaker.md`](speaker.md): media player pipeline tasks on the Connect AUX-2 with the `speaker` platform player
- [`web_server.md`](web_server.md): state SSE stress and the HTTPD task stack on web_server_idf, entity icon and device class readback, per-session `/events` listing, `/events` head, stall, Wi-Fi drop and churn checks
- [`uart_sim.md`](uart_sim.md): UART sensors simulated on a classic ESP32 with loopback and cross-wired UARTs, a Duco ventilation box model on a second UART
- [`web_server.md`](web_server.md): state SSE stress and the HTTPD task stack on web_server_idf, entity icon and device class readback
- [`uart_sim.md`](uart_sim.md): UART sensors simulated on a classic ESP32 with loopback and cross-wired UARTs
- [`gpio_expander.md`](gpio_expander.md): interrupt driven expander inputs, loop idle checks, testing a device out of reach
- [`led_strip.md`](led_strip.md): forcing a failed strip allocation, RMT symbols per chip, loop cost per strip
- [`remote_base.md`](remote_base.md): remote_receiver protocol decoding through a one-pin loopback on ESP32 and ESP8266, no IR hardware
@@ -64,11 +63,11 @@ One file per component, named after it:
- [`rs485_frame.md`](rs485_frame.md): DLE/STX/ETX bus on an ESP32-C3 with a cross-wired simulated bus device (frame cases, gate timing)
- [`hx711.md`](hx711.md): fake HX711 driven from the other core, every published value matched to the served conversion
- [`automation_trace.md`](automation_trace.md): automation codegen A/B, a marker trace of chains, conditions, loops and script modes over the API
- [`bluetooth_proxy.md`](bluetooth_proxy.md): held GATT link after a half-open API session, frozen Home Assistant stand-in, out-of-order GATT discovery responses injected on the proxy, GATT UUID forms (16 bit, 32-bit base, 128 bit) compared against an ESP32 proxy
- [`bluetooth_proxy.md`](bluetooth_proxy.md): held GATT link after a half-open API session, frozen Home Assistant stand-in, out-of-order GATT discovery responses injected on the proxy
- [`crash_report.md`](crash_report.md): crash buttons, fetching the previous boot's crash report over the API, per-chip abort facts
- [`https_idf.md`](https_idf.md): web_server_idf HTTPS test component, listener switch, stalled TLS clients, bad key
- [`mdns.md`](mdns.md): legacy unicast mDNS dump, hammer and latency straight to the device
- [`sim_buses.md`](sim_buses.md): PCF8574/HD44780, MAX7219, SPI spy, packet_interface and SN74HC165 simulators, status LED probe, fake Home Assistant, cg.static_buffer probes, a TRF7962A NFC reader chip behind a fake SPI bus
- [`sim_buses.md`](sim_buses.md): PCF8574/HD44780, MAX7219, SPI spy, packet_interface and SN74HC165 simulators, status LED probe, fake Home Assistant, cg.static_buffer probes
- [`sigma_delta.md`](sigma_delta.md): pulse density of the sigma_delta speaker pin read inside the ESP32-P4, rtttl note onset and release
- [`microphone_http_stream.md`](microphone_http_stream.md): raw PCM over HTTP with a counter fake mic or a USB mic, multi-reader client
- [`i2s_audio.md`](i2s_audio.md): I2S microphone fed by an I2S TX channel on the same S3 through three jumpers
@@ -84,7 +83,3 @@ One file per component, named after it:
- [`heap_alloc.md`](heap_alloc.md): operator new counting, and a ring of allocation callers decoded with addr2line to tell the first run after boot from repeats
- [`wled.md`](wled.md): WLED realtime sender on a board's own access point
- [`combination.md`](combination.md): combination sensor NaN behaviour from the last source
- [`improv.md`](improv.md): Improv serial with stray bytes, scan, bad and good passwords from a client that reads frames out of the log stream, Improv over BLE, per board notes
- [`arduino_pico.md`](arduino_pico.md): arduino-pico framework bump A/B: W5100S Ethernet soak, Pico W fallback AP client count, Pico W bluetooth_proxy GATT hold, web_server SSE stalls, `/events` churn crashes and never completed connects, W5100S OTA without `interrupt_pin`
- [`bluedroid_gattc_queue.md`](bluedroid_gattc_queue.md): queued MTU request and execute write recursion that overflows BTU_TASK, ble_client central against an ESP32-S3 peer
- [`tuya.md`](tuya.md): Tuya MCU simulator on a cross-wired second UART (handshake, every datapoint type, command frames logged), hub vs entity order, dump A/B
@@ -62,19 +62,3 @@ Results: https://github.com/esphome/esphome/pull/16445#issuecomment-6335435397
An API client that connects allocates about 15 times on ESP32: the connection object, then the list entities and
initial state batches growing the `DeferredBatch` vector. A log subscription adds more per line. Connect before a
heap measurement and subscribe to logs only afterwards (`heap_alloc.md`).
## Native API connect/disconnect stress (#14679, #14687, #14706)
`scripts/api/connstress.py HOST [ITERATIONS] [CONCURRENCY]` (aioesphomeapi; `API_KEY` env for a noise key) opens and
closes native API connections as fast as it can, 500 attempts with 4 at a time by default, and ends with
`RESULT PASS` or `RESULT FAIL`. It is the test that found the raw lwIP socket heap corruption, so run it for any change to
the socket layer, the scheduler or locking on RP2040/RP2350 and ESP8266, with a bounded serial capture on the device:
the client only sees resets, the crash is in the log.
- RP2040/RP2350 before #14687: crashed after 3 or 4 iterations (`malloc` from the lwIP accept IRQ while the main loop
was in `malloc`: double free in `~APIFrameHelper`, `_malloc_r` in `APIConnection()`); after: 500/500. #14679 first
added the lwIP locking between the lwIP callbacks and the main loop on RP2040.
- ESP8266 before #14706: crashed within about 50 iterations (`umm_malloc_core` LoadStoreError, use after free of a
closed PCB's callbacks); after: 490/500, the 10 failures are connection resets from overload, which is expected there.
- ESP32 (BSD sockets) passes 500/500 on both and is the control.
- Compare against dev on the same board; count reboots and decode crash addresses as well as the client's failures.
@@ -1,42 +0,0 @@
# arduino-pico framework bumps (RP2040/RP2350)
Scripts: `scripts/rp2_web_ble/` (README there). Used for #22671 (6.1.1 to 6.3.2).
- Build PR and dev with a private `PLATFORMIO_CORE_DIR` (`scripts/pio_build.sh` does; see `reference/boards.md`).
Check the compile log line `framework-arduinopico @ 1.60302.0` per build.
- Wired Ethernet (W5100S EVB): 6.3.2 defines `LWIP_ASSERT_CORE_LOCKED()` as `async_context_lock_check` on the
Ethernet context (`cores/rp2040/lwip_wrap.cpp`), so unlocked lwIP calls panic there. Soak the API (200 ms template
counter, 1 s uptime, logs subscribed), load pages, OTA and reboot; the rp2 crash handler shows a panic as
`Crash (HardFault)` in the debug reset reason on the next boot.
- Fallback AP client count: `ap_clients_` is protected and `WiFiComponent` is final; read it from an
`esphome: includes:` header through an explicitly instantiated member pointer (`appeek.hpp`). Join raises it to 1.
- bluetooth_proxy on the Pico W: `rp2_ble_tracker:` + `bluetooth_proxy: active: true`. Drive with `gatthold.py`
against the ESP32-S3 server in `scripts/ble_notify/server.yaml`. UUID forms and the 0x3E connect failures are in
`bluetooth_proxy.md`. A "Failed to send notify data response" warning at each disconnect is normal.
- web_server SSE on the W5100S (dev 2a6a1a050e4, 6.1.1): every `/events` stream stops after 0.3 to 11 s (alone, with
the API, or with page loads alike), then the EVB closes it with a FIN 5.8 to 6.3 s after its last data segment, so
curl exits 0. The pcap shows every byte ACKed by the host, window 65535, no retransmits, nothing sent in the gap:
the device simply stops writing. The 5 s gap matches RPAsyncTCP `ASYNC_MAX_ACK_TIME` (likely `_pcb_busy` not
cleared, then `_onTimeout` closes). A new `/events`, GET / and the API keep working. Pico W: 60 s steady. On macOS
`tcpdump` runs without sudo once your account is in the bpf group. Tools: `sserun.sh HOST TAG SECS alone|api|pages
N` (pcap, netstat poll, API VERBOSE log, probes) and `pcapsum.py PCAP DEVICE_IP` (per connection last data, FIN/RST,
retx). Confirmed with counters patched into the build's `.piolibdeps` copy of RPAsyncTCP: every stall is one
`_sent()` underflow with `_tx_unacked_len` 0 and the pcb idle after `send()`; ESPAsyncWebServer 3.12.1 does not
help. The esphome-libs RPAsyncTCP fork (`esphome-1.3.2`, 8efb83c) streams 16 x 30 s with no stall. `stallctr.py PCAP
IP APILOG` lines the counters up with each stall.
- `H=<device ip> churn.sh TAG SECS` (4 loops of 0.2 to 1.5 s `/events` connections) crashes the rp2 web_server: dev on
the EVB, the fork build on the EVB and the Pico W. HardFault in `AsyncEventSource::send` from
`deferrable_send_state`, about one reboot every 10 to 20 s. Ten quick reboots put the board in safe mode (ping and
OTA port 2040 only); OTA a build to recover.
- **Never flash the W5100S EVB a build without `interrupt_pin`** unless you can reach BOOTSEL: arduino-pico then polls
the W5100 from an async_context timer that every lwIP call pushes back, so the esphome OTA read loop starves the
poller and OTA never completes (the device ACKs the hello, then never the features byte). Web and API still work.
The board has no USB CDC port with ESPHome firmware, so the only way back is BOOTSEL and the UF2 drive.
- `churnflow.sh` (in `rp2_web_ble/`) repeats `churn.sh` with heap and main loop checks and reflashes by OTA when a
round left the board in safe mode.
- `leakflow.sh` with `neverconn.py HOST N rst|head`: ESPAsyncWebServer 3.9.6 leaks about 175 B per `/events` client
that resets before the response head is ACKed (52 KB per 300); 3.12.1 stays flat.
- `/events` churn also hangs the BK7238 main loop or drops it off the network on dev; recover with a serial reset.
- The API has 3 sockets on these builds: connstress at concurrency 4 always shows some "dropped after hello" failures;
use 2 for a clean 500/500.
- IRQ context scheduler test for #22688: `scripts/rp2_sched_lock/`, described in `scheduler.md`.
@@ -1,47 +0,0 @@
# Bluedroid GATT client command queue
## Queued MTU request recursion overflows BTU_TASK (#17954, #22670)
Bluedroid's `bta_gattc_pop_command_to_send()` runs the head of `p_cmd_list` and removes it only after
`bta_gattc_sm_execute()` returns. When that head is an MTU request (or an execute write) whose send fails at once,
the failure path pops again with the same command still at the head, so it recurses until the BTU task stack
overflows. Every ESP-IDF release so far; PR #22670 fixes it in ESPHome with linker wraps
(`bluedroid_gattc_queue.c`).
Rig, `scripts/bluedroid_mtu_recursion/`:
- Peer: an ESP32-S3 with `btusrv.yaml` (`esp32_ble_server`, one readable and writable characteristic, advertising).
- Central: a classic ESP32 devkit with `btu-common.yaml` plus a per-build file like `btucli-example.yaml` (set
`peer_mac` to the peer's Bluetooth MAC, which is its Wi-Fi MAC + 2, and give each build its own name).
`CONFIG_FREERTOS_WATCHPOINT_END_OF_STACK: y` makes the overflow panic at the write:
`Stack canary watchpoint triggered (BTU_TASK)`.
- Each connection: `on_connect` looks up the characteristic handle right away (a `ble_client` with no child nodes
releases services and cleans the GATT cache in the same SEARCH_CMPL event, so a lookup after a delay finds
nothing), waits 1.5 s, then suspends `BTU_TASK`, calls `esp_ble_gap_disconnect`, queues 4 reads and one MTU
request, waits 30 ms and resumes. The batch is the key: sent one at a time, the BTU task handles each request
before the next arrives and nothing waits in the queue. The reads fail and keep the command slot busy so the MTU
request is queued; when popped, `GATTC_ConfigureMTU` fails with "BT_GATT: connection not established".
`ble_client` reconnects on its own and the next round starts.
- `analyze.py CAPTURE` counts rounds, BTU canary crashes, boots, reconnects with and without a reboot and MTU 517.
- Decode with the IDF `xtensa-esp32-elf-addr2line -pfiaC`; the overflow shows the repeating
`pop_command_to_send (act.c:1739/1746) -> sm_execute -> cfg_mtu (act.c:1042)` frames.
Results on ESP-IDF 5.5.6: dev at 46844d3631 crashed 73 of 73 rounds in 5.5 min (every round, about 4.5 s per
reboot cycle); the PR at ad787163fd ran with no crash and reconnected without a reboot each time. Check the wraps are
linked with `nm firmware.elf | grep __wrap_GATTC_ConfigureMTU` (native ESP-IDF builds only; the define is
`USE_ESP32_BLE_BLUEDROID_GATTC_QUEUE`).
### Execute write variant
`btuxw-example.yaml` sets `exec_write: "true"`, which sends `esp_ble_gattc_execute_write(gattc_if, conn_id, true)`
in place of the MTU request. No prepare write is needed: Bluedroid does not check for prepared writes, and
`GATTC_ExecuteWrite` fails synchronously with the same "connection not established" error. The variant also sets
`CONFIG_BT_LOG_GATT_TRACE_LEVEL_API: y`, so each send logs `BT_GATT: GATTC_ExecuteWrite conn_id=... is_execute=1`,
and those lines are counted per round with awk between the `ROUND n start` lines. In a few rounds (about 3 to 14%)
the controller reports the disconnect before the command is popped. Those rounds log no execute send and never
crash, so count "rounds that reached the send", not all rounds. Most dev backtraces in this variant are cut off in
`vTaskSwitchContext` (the extra trace logging makes task switches more likely at the moment of overflow). Use the
complete ones to decode the `bta_gattc_execute (act.c:1352)` cycle.
Results: dev 46844d3631 had 81 rounds, 70 reached the send, and all 70 crashed. PR e51f0b8321 had 151
rounds, 147 reached the send with exactly one execute line each, and there were 0 crashes and 1 boot.
@@ -55,38 +55,3 @@ real device uses, decode an Android HCI snoop: a Read By Type response lists its
- Give the proxy an `api` encryption key (`API_KEY` for the scripts). A Home Assistant on the same network connected to
a plaintext test proxy and took over its Bluetooth subscription (`Subscription from Home Assistant ... replaces
aioesphomeapi`), dropping the test client's connection mid-run.
## GATT UUID forms through a proxy (#22679)
`scripts/gatt_uuid/`: `server.yaml` (the `ble_notify` server plus a service with 128 bit, 32-bit base and 16 bit
UUIDs on characteristics and descriptors, and 2901 and 2902; set the name with `-s name`), `gattlist.py PROXY PEER
OUT.json` (lists every UUID, reads, notifies 10 s, retries 0x3E), `cmp.py REF.json OTHER.json` (entry by entry diff,
exits 1 on a difference) and `run.sh NAME` (Pico W proxy from `rp2_web_ble/picow.yaml`, list, compare). Take the
reference from an ESP32 proxy (unset `API_KEY` for a plaintext API).
- On rp2 before #22679, BTstack truncated 32-bit base UUIDs with a nonzero low half to 16 bits (`5ec10001-...` came
out as `00000001-...`); a zero low half (5ec10000) came through intact.
- Connects through the Pico W proxy fail with 0x3E now and then (see `reference/boards.md`); the scripts retry.
## Scan complete never arrives (#22770)
`scripts/ble_scan_complete/`: `scan.yaml` is a proxy config like the report's (ESP32-C6, interval = window = 1100 ms,
debug sensors, `api: reboot_timeout: 0s`, a `duration` substitution, 5 s by default; the validator wants at least
three intervals, so 2 s is refused). The tracker watchdog is `duration * 2`, so a short duration shortens it too.
`scaninst.yaml` adds `scan_inst.hpp` and five `-Wl,--wrap` flags that count one scan's duration timer through
Bluedroid: `BTM_BleScan` starts and stops, `btu_start_timer(BTU_TTYPE_BLE_SCAN)`, the timer's alarm firing (a shim on
its `osi_alarm_new` callback, BTC task), `btm_ble_timeout` (BTU task) and the INQ_CMPL handed to `btc_transfer_context`
(PID 5, act 3; the param is 108 bytes in libbt, so do not compare with the C++ `sizeof`). A 30 s interval logs them as
`scaninst`; a lost completion shows where the chain stopped (timer > fire + 1, fire > timeout, timeout > cmpl_post, or
cmpl_post above the ESPHome-side `ends / 2`). `ha.py HOST OUT SECONDS [--toggle S --jitter J]` is the Home Assistant
stand-in (raw adverts, scanner state, set mode passive once, optional active/passive toggles). `soak.sh TAG HOST
SECONDS [ha.py options]` (env PORT, API_KEY or else `apikey.txt`) runs a bounded serial capture plus ha.py.
`wait.sh CAPTURE SECONDS [FROM_LINE]` blocks until a timeout, drop or unexpected-state line and prints the counters.
- Bluedroid and IDF log lines reach the ESPHome log as tag `esp-idf` at VERBOSE (`[V][esp-idf:000][BTU_TASK]: E ...`),
so a user capturing at INFO over the API never sees a Bluedroid error.
- A stop from Home Assistant landing just after the timer fired (toggle period 5.03 s with 0.08 s jitter against 5 s
scans hits it about once per 20 minutes) gives `Unexpected state: STOPPING on scan complete`, `stop complete -
status 1`, `BTM_BleScan scan not active`; the tracker recovers. A stop while STARTING logs `Cannot stop scan:
STARTING` and the mode change waits for the next restart.
- With about 55 devices around (about 95 adverts/s), no lost completion in about 2800 natural 5 s cycles on 2026.10.2
(VERBOSE and INFO builds) plus about 670 toggled stops, and none on dev.
@@ -24,11 +24,3 @@ put `scripts/` on `PYTHONPATH`):
- Quick crash reboots count toward BK72xx safe mode, so space out back-to-back crash tests on those boards.
- A device that keeps the previous crash replays it to every new log client until the next crash; that is one
record, not two.
## Crash records and log subscribers
The record is replayed and cleared by the first log subscribe (`on_subscribe_logs_request`), so drain it on the
first boot after an OTA, or a stale record from the previous build shows up. Moving `api: port:` keeps Home Assistant
off the board while testing. `scripts/crash_report/quiet_then_subscribe.py HOST PORT OUT N` drains the record, stays
connected without a log subscription for N reboots, then subscribes and records the replayed record; it tells a crash
that needs a log subscriber (like the W5500 IPv6 reboot abort in `reference/boards.md`) from one that does not.
@@ -86,15 +86,3 @@ Notes:
them from the device's `Got state` log lines.
- Turning a script off stops it mid run (the flag stays on). A script that finishes at once reports on, then off, in
the same second.
## Subscription order and dump order (#22725)
- `hadrive.py` also prints `<phase> SUBORDER NNN <line>` in the order the device sent the subscriptions (the SUB lines
are sorted). `grep -v SUBORDER drive.txt` diffs the sets, `grep SUBORDER` the order.
- Dump block order per build: `python3 scripts/lognorm.py boot1.txt | grep -a "^\[C\]\[" |
sed -E 's/^\[C\]\[([a-z_.0-9]+)\]: (.{0,40}).*/\1 | \2/' | awk -F' [|] ' '{ if ($1!=p) {print $1 " :: " $2; p=$1} }'`
prints one line per component run; diff dump text between builds through `lognorm.py` too, which drops the
`:NNN` line numbers.
- Idle ESP8266 free heap in this config swings by 2 kB between readings; compare the steady reading (the fifth one
after boot) per boot, which matched the heap_size change exactly (+168 B for 15 import entities on #22725).
- The wizard light (`l_wiz`) needs #21908; a stack based before it needs it dropped on both builds (`NO_LIGHT=1`).
@@ -1,46 +0,0 @@
# Improv serial and BLE (#22597)
`scripts/improv/` drives Improv on a freshly erased board with no saved network:
- `improvcli.py PORT OUT --secrets FILE`: Improv serial client. It shares the console with the logs and parses
frames out of the log stream. The flow: 300 stray bytes with no `I` in them, state, info, network state, scan, a
bad password (expect `unable_to_connect` after about 30 s), the real password, then state and network state
again. It ends with a count of frames, bad frames, port resets, reboots and log lines. Credentials come from the
secrets file and are never printed, and the SSID is masked in scan results. It reopens a native USB port that
re-enumerates. A reboot is counted from each new `Running through setup()` log line after the settle, so a board
behind a USB-UART bridge that reboots without a port reset is caught too.
Every request must get its reply; a missing reply, an unexpected error, a malformed frame, a port reset or a
reboot fails the run. `--skip-bad` leaves out the bad password case, `--keep-dtr` leaves DTR asserted,
`--allow-reset` accepts a reboot.
- `improvble.py NAME SECRETS`: Improv BLE client (bleak). Finds the device by name, sends WIFI_SETTINGS and prints
the state changes and the RPC result URLs. Needs `improv_ble` (formerly `esp32_improv`) with `authorizer: none`
and a `next_url`; the result must carry a URL.
- `common.yaml`: package with Wi-Fi and no network, `web_server`, `improv_serial` with a `next_url` (so the
provisioned reply carries URLs) and an ALIVE line every 5 s. `example-esp32.yaml` uses it on a classic ESP32.
- `run.sh NAME [improvcli args]` (env `TREE`, `PORT`, `SECRETS`, `PY`): compile, erase, flash and run the client.
It erases with esptool, so ESP chips only; on LibreTiny or RP2040 erase and flash with that platform's tools and
run `improvcli.py` directly. Both clients end with PASS or FAIL and exit non-zero on FAIL.
Per board:
- Improv serial uses the logger's UART, so the client must open the port the log is on. On LibreTiny check which
UART `logger: hardware_uart: DEFAULT` resolves to (see the board notes).
- BW15: drop `web_server:` or add an `ap:` (it resets during setup otherwise), and pass `--allow-reset` for the known
reboot after `Disabling AP`; the provisioned reply usually goes out first, not always. See `reference/boards.md`.
- ESP32-C3 on USB_SERIAL_JTAG: passes with and without `--keep-dtr`; its reboots only show in the
`Running through setup()` count (see `reference/boards.md`). The scan returns only the last scan's results, which
are empty in the first seconds after boot; use `--settle 10` or more when the target network should be listed.
- Pico 2 W: run with `--keep-dtr` (see `reference/boards.md`). Its Improv scan lists no networks, on dev and PR alike.
- Erase the flash between builds: a saved network skips the provisioning path entirely. Both clients fail at once
when the board already reports `provisioned`; erase the flash first. A new config name does not help: the
network is saved under a fixed key when the config has no `networks:`. A provisioned board also stops advertising
Improv over BLE, so `improvble.py` reports the device as not found.
- If the real password fails right after the bad password case, rerun once before blaming the change: on the C3 it
failed once (`Auth Leave` while Wi-Fi was still retrying the bad network) and passed on every rerun.
When posting results:
- Scan results list the neighbours' SSIDs; filter `RPC cmd=4 len=[1-9]` lines out of anything you post.
- Mask IP addresses in the result URLs.
- Cover a long password too: provision once against a network whose password is 20 characters or more (the client
prints the length).
@@ -61,10 +61,3 @@ frame of two LEDs set to `Color(0xA5, 0x3C, 0x0F)` reads `3ca50f3ca50f`.
- An RP2350 `panic()` escalates to a HardFault and the rp2 crash handler reboots within about a second, so a panic
during component setup boot-loops before USB CDC enumerates and the board looks dead (no port, no BOOTSEL drive).
Hold BOOTSEL while plugging it in and flash the UF2 drive. To see such a fault, trigger it after boot.
## Counting a PIO strip's edges on its own pin (#16243)
A GPIO interrupt on the strip's own data pin (`gpio_set_irq_enabled_with_callback(pin, GPIO_IRQ_EDGE_RISE, ...)`)
counts every bit the PIO program sends, with no wires or LEDs; a strip that sends nothing shows 0 edges per report.
Log the PIO block's state machine enable mask and program counters next to the count. `pio_probe/pio-common.yaml`
clears PIO1 instruction memory at boot because it survives a reflash (see `reference/boards.md`).
@@ -34,11 +34,3 @@ wave on the same pad no longer reaches the input (the pad reads 0).
- Result on #21074 at 2ef417bd81 with peripherals powered down: the PR caught all 644 real edges (dev 66) but added
4642 calls while the line was high, one per sleep cycle (about 16 ms), with the level read correctly. Peripherals
on: both builds caught every edge. https://github.com/esphome/esphome/pull/21074#issuecomment-6503939850
## Power lock check
`power_management: esp32_power_management` with `profiling: true` and an `interval` lambda that calls
`esp_pm_dump_locks(stdout);` prints the lock and mode tables on the USB console (not through the ESPHome logger).
Compare lock lists between builds: `usb_serial_jtag` holds `NO_LIGHT_SLEEP` by itself. For a soak that the USB console
must not keep awake, set `logger: hardware_uart: UART0`, dump the locks into `open_memstream()` and log the SLEEP,
CPU_MAX and light sleep count lines, read over the API.
@@ -1,29 +0,0 @@
# nPM1300 PMIC on the Seeed XIAO nRF54LM20A (#22661)
Scripts: `scripts/npm1300/`. The board's nPM1300 sits at 0x6B on P1.18 (SDA) and P1.17 (SCL); board facts, including
which rails must never be switched off, are in `reference/boards.md`.
- Probe first: `npmprobe.yaml` has the hub with no regulator or charger settings, scans the bus and dumps the
registers every 20 s, so you see what is on before any setting is written.
- `npm-all.yaml` (with `npmcommon.yaml`) has every sensor, binary sensor, text sensor and switch, logs each value from
`on_value` and `on_state` lambdas (entity state lines are VERBOSE on dev, so DEBUG shows nothing without them), dumps
the registers and switch states, and toggles the charger and LDO1. Put a setting under test (the VBUS limit, the
charger values) in `npmcommon.yaml`, not in the per-run file.
- `regdump.yaml` reads a fixed register list with `id(pmic).read_register16()` and logs `REGS 0201=.. ...`; compare two
builds' dumps after stripping timestamps and source line numbers.
- `runone.sh NAME CONFIG PORT [SECS]` flashes with `esphome run --no-logs` (pyOCD) while a serial capture stays open on
the same port. Builds without `ota:` have no MCUboot.
Findings:
- With no battery and the charger on, VBAT reads about 4.23 V, IBAT about 0.2 mA, battery detected is ON and the status
flips between Trickle Charge and Constant Current; with the charger off it says No Battery, and it can drop to Not
Charging for one update on its own. The battery temperature stays at 24.6 C (a fixed 10 k on the NTC pin). Charge
current scale and sign need a real cell.
- ADCIBATMEASSTATUS (0x0510) reads 0x0C (charge) most of the time with no cell and the charger on, sometimes 0x08 or
0x0F.
- `vbus_current_limit: 300mA` writes ILIM0 and ILIMSTARTUP as 0x03 and the board stays up on USB.
- BUCKSWCTRLSEL (0x040F) keeps its value across reflashes until a power cycle, so a setup write to it shows no change on
a warm board.
- To prove a setup priority change, add `on_boot` hooks at 1001, 950 and 900 that log markers and see where the
component's first publishes land between them.
@@ -44,116 +44,3 @@ only the last line sees "Failed to upload" first, so rerun before blaming the bu
output. `scripts/esp8266_lock_load/ir-one-pin.yaml` does this, and its `run.sh` adds Wi-Fi load (see
[one_wire.md](one_wire.md)). Under load, the build that sends the whole frame in one lock hit a Hardware Watchdog in
`wDev_ProcessFiq` after 11 minutes.
## Climate IR loopback over the API (#22704 whirlpool, #22709 climate_ir)
- Pattern: one image with a sending climate and a receiving climate (`receiver_id`) per platform on one shared pin, an
aioesphomeapi script that sends climate calls and reads every climate's state after each step. Several platforms can
share one receiver; each rejects the others' frames. Scripts in `scripts/climate_ir_loop/`: `drive.py`
(steps `Name:mode=cool,t=24,fan=low,swing=vertical,preset=eco` or `svc:action?flag=1`), `run.sh NAME PORT [STEPS]`
with `steps.sh` (prelude and steps), `frames.py NAME` (splits the `dump: raw` lines per step), `compare.py A B
[TOL] [--flat]` (per step, or with `--flat` the whole sequence timing by timing), `base.yaml` (network, API and
logger), `c-common.yaml` (Samsung, Friedrich, Siemens IRA211 and Mistral senders and receivers), `esp32-ir.yaml`
(GPIO27) and `esp8266-ir.yaml` (GPIO12) as one-pin loopback board packages (`pin`, and on the ESP32 `rmt_symbols`
and `receive_symbols`, are substitutions; the Pronto configs include them too), `c32pr.yaml` (an example run file)
and `whirlpool-inject-common.yaml` (injected frame actions).
- ESP32-PICO-D4 kit: GPIO4 does not loop back cleanly (every mark reads 162 µs short, a -162 space leads the frame, so
nothing decodes). GPIO27 gives exact timings (572/553/1659 µs). Try another pin before suspecting the code.
- RMT ends a frame on any level longer than `idle`, mark or space. A whirlpool 14 byte frame ends after a 7,960 µs gap,
so its decoder needs the trailing idle to be within 25% of that, but the 9 ms header mark must stay below idle:
`idle: 9500us` works, 8500 cuts the header off.
- Restored climate state changes the first frames (power toggles, Samsung power section): start every run with a
prelude that puts each sender in a fixed state (every field each sender supports, such as humidity for daikin_arc
or a fan mode daikin_brc supports), and compare only the frames after it. ESP8266 restore lives in RTC memory, so a
receiver's state can carry over from the previous image.
- Inject frames a sender cannot produce (an out of range temperature code, a short frame) from an API action lambda with
`item_bytes<BitTimings>` and `call.perform()`.
- Assigning frames to steps by host timestamps is fragile on the D1 mini (a log line can land after the next step
starts); compare the whole frame sequence in order instead. Two frames sent back to back (siemens_ira211 power plus
sync) come out as one dump, and the gap between them varies by up to 1 ms on ESP8266.
- Samsung power frames are 347 timings: an ESP32 receiver needs `rmt_symbols: 384` and `receive_symbols: 384`.
## Protocol A/B with `dump: all` (#22711)
- One button per `transmit_<protocol>` action, one binary sensor and one `on_<protocol>` trigger per protocol, each
logging a marker; count markers and compare the dumper lines of the dev and PR runs in order. All of nec, dooya,
nexus, aok and cardin_s438 decode in the ESP32 GPIO27 loopback with the default idle.
- Cardin S438 picks a random seed per frame (`random_uint32()`), so its pronto and cardin lines differ between any two
runs; compare code and channel only. Each press sends 3 repeats, so its trigger and binary sensor fire 3 times.
- Start the dump extraction at the first `MARK tx` of any protocol: the API lists buttons by name, not config order.
- A Python-only change can still move generated code: diff `src/main.cpp` of the two builds (field order in action
setters, defaults now written explicitly).
## Large climate frames in the loopback (#22710)
- At 115200 baud the serial log cannot keep up with long `dump: raw` frames (hitachi_ac344 is 691 timings, daikin
sends 3 frames per command). Lines get lost or merged, frames show up under the wrong step, and the script reads the
receiver state before it has decoded the frame. Use `logger: baud_rate: 921600`, capture with `--baud 921600`, and
wait 4 s per step (`BAUD=921600 STEP_S=4 run.sh ...`). A CH340 D1 mini can garble 921600; 460800 carries the long
frames cleanly there.
- On the classic ESP32 the receiver has no RX ping-pong, so a whole frame has to fit `rmt_symbols`. With a 64 symbol
transmitter, `rmt_symbols: 448` and `receive_symbols: 448` hold hitachi_ac344.
- Receivers that keep state the frame does not carry (climate_ir_lg swing is a toggle, zhlt01 keeps its preset on
"none") start each run from restored state, so compare them without those fields or erase the preferences first.
## Whirlpool A/B by hex lines (#22710 retest)
- whirlpool logs every frame at VERBOSE (`Sending:` and `Received:` hex), so on the D1 mini compare those bytes line by
line instead of raw timings. One image per model (DG11J1-3A, HITACHI_HCRA31NEWH, HISENSE_ARG; "Hisense ARG" is a
whirlpool model, not its own component). `scripts/climate_ir_loop/whirlpool/`: `cl.yaml` (the model
as a substitution, combined with `../base.yaml` and `../esp8266-ir.yaml`), `steps.sh`, and `all.sh PORT BASE_TREE PR_TREE`, which builds and runs base then PR per model so the
board ends on a PR build.
- The HISENSE_ARG receiver reports fan medium for fan_only while the sender says auto (dev too).
## Pronto dump A/B with `dump: all` (#22752)
- Scripts in `scripts/remote_base/pronto/`: `gen.py` writes `gen-common.yaml` (buttons for NEC and raw frames of
21/41/43/121/245 timings, a soak button with heap logs, a `pronto:` binary sensor), `esp32-ir.yaml` (GPIO27) and
`d1-ir.yaml` (GPIO12) include it with the board packages from `climate_ir_loop/`, `run.sh NAME TREE YAML PORT BAUD
ROUNDS` flashes, captures, presses every tx button ROUNDS times and then the soak button with `../press.py`, and
counts the markers, triggers and `on_press` lines on serial; `extract.py` keeps the dump lines without timestamps
and source line numbers, `consist.py` checks every dump against the `on_pronto` text, `jit.py` counts frames that
differ between two runs.
- The received frame ends with the idle gap as an extra timing (`0181` for the 10 ms idle), so a `pronto:` binary sensor
built from the sent timings needs one more number at the end, or the length check rejects every frame. A 41 timing
frame is then exactly 230 characters, one full log chunk: a good boundary case.
- ESP32 RMT loopback gives byte-identical Pronto text across builds. The D1 mini edge interrupt moves timings by one
Pronto unit between any two runs (about half the frames), so compare D1 runs by dump against trigger consistency.
- `dump: all` prints NEC lines between the `on_pronto` trigger output and the Pronto dump; the API binary sensor
ON/OFF pair can arrive collapsed, so count `on_press` log lines on serial instead.
## Protocol decode A/B on the host, climate fuzz, AEHA/Haier/Mirage loopback (#22773)
- `scripts/remote_base/hostharness/`: a `host:` config (`rbhost.yaml`) with the local `rbtest` component in
`scripts/remote_base/ext/` (AUTO_LOAD remote_base, `request_protocol()` for every protocol source) and `rbh.hpp` called from `on_boot`; it encodes, mutates, decodes
and dumps, then `exit(0)`. Run the `program` binary and compare stdout through `scripts/lognorm.py`. The Haier
and Mirage decoders want one more item after the last mark (the receiver's idle gap): append `-20000` to transmitter
timings or every clean frame fails.
- AEHA frames longer than 35 bytes decode as their first 35 bytes (the trailer mark equals a bit mark).
- `scripts/climate_ir_fuzz/`: copy `climate_fuzz.hpp` plus the matching `*_fuzz.cpp.example` (as `.cpp`) into
`tests/components/<climate>/` of each tree (haier_tundra also needs `haier_init.py` as `__init__.py`), run
`script/cpp_unit_test.py`, diff the `FUZZ` lines. `run()` appends the receiver's idle gap to every frame, and
`aeha_random()` builds the random frames for the AEHA climates. All tests of one run share one main.cpp: use a macro include guard,
not `#pragma once`, for a header copied to several test dirs.
- Lambda idiom checks: compile `void f(AEHAData x) { ... }` snippets with `-fsyntax-only -I<host build>/src` and
`#include "esphome.h"` (format_hex lives in alloc_helpers.h).
- Loopback config `ir-common.yaml` (AEHA 2 to 35 bytes, Haier, Mirage from an API button, `../press.py HOST tx_all:14 ...`;
the trigger lambdas call `rbh_on_frame()` from `rbh.hpp`, so keep the run file next to it). An ESP32-PICO-D4
on GPIO27 never decodes 34 or 35 byte AEHA, even with `rmt_symbols: 384`, `buffer_size: 8kb` (dev too); a D1 mini on
GPIO12 decodes all of them.
## Every dumper over a saved corpus, on the host (#22814)
`scripts/remote_base/dumpall/`: the same `rbtest` component requests every `_PROTOCOL_STEMS` source; `rbh.hpp` needs a
`dumpers.h` that `gen_dumpers.py TREE` writes from the `DECLARE_REMOTE_PROTOCOL(...)` lines of the tree (plus Pronto,
RCSwitch, ABBWelcome; Raw only when nothing matched). Encoders that randomise (Cardin S438 picks a random seed) make two builds see
different inputs: run once with `RBH_CORPUS=<file>` to write the corpus, then replay that file on both builds.
Merge back-to-back marks or spaces before decoding (Drayton never decodes otherwise). Nudging a quarter of all
timings to the tolerance edges proves nothing (the frame fails elsewhere): put only the sync mark at lo-1, lo, hi,
hi+1 and keep the rest exact; that variant caught a planted `< max` in 1,000 decodes per protocol.
`cmp.sh` compares the DEBUG and VERBOSE outputs (`rbhost.yaml` built with `-s level DEBUG` and `-s level VERBOSE`)
through `scripts/lognorm.py`.
- Loopback with `dump: all` (`ir-common.yaml`, `../press.py HOST tx_all:9 tx_all:9 verbose tx_all:9 debug`): Beo4 never decodes on one pin (needs its carrier) and
Drayton's channel/command form has no checksum; compare decoded lines, raw and Pronto timings jitter between runs.
- `cc-common.yaml`: an ESP32-C3 with a CC1101 in async OOK at 433.92 MHz, GDO0 into remote_receiver, for the RF
dumpers on real air (a quiet band gives only a few Pronto dumps a minute and no protocol hits).
@@ -27,13 +27,3 @@ captures 40 s from boot and runs `ivsum.py`, which prints the run count, first r
10 s after setup the scheduler frees its item pool and shrinks its `to_add_` and `items_` vectors
(`SCHEDULER_FREELIST_TRIM_DELAY_MS` in `application.cpp`). The first burst of timers after that allocates new
items and regrows the vectors once, so heap or timing measurements should skip the first run after boot.
## RP2 scheduler lock under IRQ and lwIP load (#22688)
`scripts/rp2_sched_lock/` (README there). The `irq_sched_stress` external component calls `defer()` and `set_timeout()`
from a 300 us pico-sdk repeating timer while the main loop does the same; the removed item race integration fixture
runs on the board as an API action. `flow.sh` runs the actions, API connect churn (`../api/connstress.py`), REST POST
churn and both together, then the actions again, and fails on a failed action, a failed connstress 500x2 or a reboot.
Judge by counts, not order: RP2 is `ESPHOME_THREAD_SINGLE`, so `defer()` order is not FIFO on any build. Keep
`debug_scheduler` off, it logs from the IRQ. Dev before #22688 failed the defer count on the Pico W; the PR passed.
`looptime.sh` compares the 16 ms interval gaps with the API and page loads running between two builds.
@@ -73,16 +73,3 @@ address they do not simulate.
- An `addressable_light` display writes nothing until its light is turned on.
- Before calling a rebased PR's change trivial, diff the PR's own diff (head against its parent) old against new: a
move that looks like a merge only can carry real commits from below.
## TRF7962A fake chip (#22188)
`scripts/trf7962a_sim/`: `trfsim.hpp` replaces the SPI component's bus (`SPIComponent` is final, so `spi_bus_` is
reached through the explicit-instantiation member pointer trick) with a `FakeBus` from `on_boot` priority 800, between
the bus (1000) and the reader's setup (600). The bus hands both of the reader's clients (MODE0 write, MODE1 read)
delegates on one register, FIFO and IRQ model; the IRQ pad is switched to input/output after setup and driven by the
model. `trfpr.yaml` runs the tag scenarios (none, present, slow, flaky, swap, collision, CRC, error flag, bad MSB,
TX only, stuck IRQ) and prints per-phase counters (CS, field timing, TX payload, reads without an address). The
same FakeBus approach works for any SPI chip whose reply depends on the address byte inside one CS window, which the
ESP32 SPI slave cannot fake.
The header targets the SPI API of the #22188 branch (`SPIBus` and `SPIComponent::spi_bus_`); dev has since dropped
`SPIBus`, so on a newer tree the bus hook has to move to the new delegate creation before it builds.
@@ -1,30 +0,0 @@
# Tuya MCU simulator (#22724)
No Tuya device needed. `scripts/tuya_sim/`: `tuyasim.hpp` is a Tuya MCU on a second UART of the same ESP32, cross-wired
through the GPIO matrix to the hub's UART (hub tx 9 rx 10, MCU tx 10 rx 9 on an ESP32-PICO-D4, `board: pico32`,
`allow_other_uses: true` on all four pins, 9600 baud). It answers heartbeat (0x00 first, then 0x01), product query,
conf query with an empty payload (so the hub takes the WIFI_STATE path), wifi state and datapoint query (one report with
all 14 datapoints), echoes each DATAPOINT_DELIVER back as a report like a real MCU, and logs every hub frame as
`SIM RX #n cmd XX: <hex>`. `tuyasim::start()` (the "Sim start" button) plays 18 scripted steps 1.5 s apart: every
datapoint type (bool, value incl. negative, string, enum, 1/2/4 byte bitmask, raw), a four datapoint frame, one
DATAPOINT_REPORT_SYNC (the hub must ack with 0x23), a repeated value.
- `tuya.yaml`: hub plus binary_sensor, switch, three sensors (int, enum, bitmask), three text sensors (string, enum,
raw), three selects (enum, string, int), water heater (switch/target/current/mode), number and fan. Hub
`on_datapoint_update` lambdas log `HUB dpN x=.. <entity>=..` with the entity's state, and entity triggers log
`ENT <id> <value>`, so the hub-before-entity order and the old state the hub sees are visible per datapoint.
`name: ${name}`, built with `pio_build.sh compile <name>` per tree.
- `tuyadrive.py HOST OUT.json` (API_KEY env): subscribes states, presses Sim start, then sends switch, select,
water heater (on, target, mode eco/electric, off), number and fan commands 2 s apart; prints per phase counts.
- `run.sh NAME TREE` (env PORT, API_KEY): upload, serial capture with a reset (stopped after the drive), drive.
- `compare.sh A B`: normalises both logs with `../lognorm.py` and diffs API states, the entity dump block, the 1 s re-dump, the HUB/ENT order and the non-heartbeat
frames the simulator received. Heartbeats move with timing, so they are left out.
Facts from the run:
- At init the fan delivers its own state (`1f01000100`) before DP31 is known and the hub logs `Setting unknown
datapoint 31`; same on dev.
- The API coalesces states published in one loop pass: the four datapoint water heater frame gives one state.
- A switch or select command for the value it already has sends no frame.
- Water heater target 55 with multiplier 0.5 is sent as 110 (`15 02 0004 0000006e`); eco/electric as enum 0/2 on DP23.
- #22724 changed only the dump order (children now print right after the hub's own lines); `setup_priority:` on a
tuya sensor is rejected on the PR and accepted on dev.
@@ -72,23 +72,3 @@ case, and `T1 X` shows which frames got through.
about 49 s after reset. Only the second boot logs `CASE done`, so a skipped reboot fails the run.
See `rs485_frame.md` for the same cross-wired UART trick on an ESP32-C3.
## Duco ventilation box (#22560)
`scripts/uart_sim/duco/`: `ducosim.hpp` is the box model from `tests/integration/fixtures/uart_mock_duco.yaml` (ack
plus answer, AA 01 stuffing, CRC16) on a second UART of the same board; on an ESP32-C6 the hub used tx 18 rx 19 and
the box tx 19 rx 18 at 57600, wired across. `gen.py NAME pr|dev` writes the config (every text sensor, sensors, mode,
comfort, sntp clock) with an `on_value` log per entity (`STATE id = '...'`, newlines as `\n`); `DUCO_VERBOSE=1` shows
`Found node`, `DUCO_SERIAL_SIZE=18` gives the dev fixture's short serial answer. `run.sh NAME KIND SECONDS` (env
`TREE`, `DUCO_PORT`, `PY`, optional `BUILD_LOCK`), run from a scratch directory with a copy of `ducosim.hpp`,
generates, builds through `scripts/pio_build.sh`, flashes and captures with `scripts/serial_capture.py`; a pr build
ends with `check.py` against `expected.txt` (exit 1 on a mismatch), and the model logs requests it has no answer for.
- Answer one request per poll call. Answering a 26 request burst in one pass blocks about 100 ms and overflows the
hub's 256 byte UART RX buffer, which silently drops the serial and version answers every round.
- Dev's hub frame is 32 bytes: the PR model's 34 byte serial answer is dropped there and duco sets "No answer".
- Error cases 7 (three long, 210 B) and 8 (four short, 148 B) check that the code-only fallback goes by length.
## ESP8266 software serial byte checks
The worst case for an unlock after a trailing run of high bits is a frame of bytes all below 0x80; a mixed frame
hides it.
@@ -59,46 +59,3 @@ valve, update, alarm_control_panel or water_heater entity reboots with `Exceptio
`json::linked(const char *)` (the ROM `strlen` on a flash address), fixed by #21316. The cover in the template keeps
that path covered; before #21316 is merged, apply its one line change to both trees or the web_server comparison
cannot run.
## Basic auth on the async web servers (ESP8266, RP2040, LibreTiny)
`scripts/web_auth/` (see its README) fuzzes Basic auth over raw sockets: functional routes with and without
credentials, wrong credentials, other schemes, a tampered hash, spacing and case, then large headers, many failed
attempts next to an authenticated `/events` stream, and a timing comparison of wrong first and last bytes. It also
has a captive portal client package that probes `/`, `/config.json`, POST with and without credentials and
`/wifisave`. `compare.py` lines a dev run up with a PR run. These scripts were still being extended when copied here.
The README also covers the newer flows: the heap under paced connection churn (`heap_churn.py`, the LibreTiny TIME_WAIT
check for #22696) and the main task stack high-water mark with the JSON arena (`arena/`, #22699).
## Per-session /events listing on ESP32 (#22691)
Checks that a new `/events` client's entity listing goes only to that client. Config
`scripts/web_server/sse-listing.yaml` (ESP32 ESP-IDF, web_server version 3, 19 entities, 3 sorting groups, heap logged
every 5 s); build each head with a different `-s name` through `pio_build.sh`. Driver:
`python -I scripts/web_server/sse_listing.py DEVICE_IP --rounds 4 --out events.jsonl`. Each round opens streams 1 and
2, waits for their listing, connects stream 3 and counts for 10 s per stream. A `state` event that carries `domain`
is a full detail (detail_all) entry; a `ping` with only `uptime` is the 10 s keepalive, any other `ping` the greeting
(config JSON). The greeting and `sorting_group` events always go only to the new session, so only the detail_all
count tells the builds apart. Ends `RESULT PASS` or `RESULT FAIL` (exit 0 or 1).
On an ESP32-PICO-D4 the control sent all 19 detail_all entries to streams 1 and 2 on every round, the PR none; stream 3
was ordered on both.
## /events stream rework checks (#21973)
Config `scripts/web_server/stream-dut.yaml` (ESP32 ESP-IDF, 20 ms template sensor, a "Wifi Drop" button that runs
`wifi.disable`, 8 s, `wifi.enable`, heap every 5 s). `scripts/web_server/stream_retest.py HOST MODE` has five modes:
`head` (raw bytes of the first 3 s; compare two builds with `head_compare.py A B NAME_A NAME_B`, which masks numbers
and the device name), `multi` (4 clients 20 s, events and longest gap), `stall` (a client with a 2 KB receive buffer
stops reading for 35 s; read the close time from `Closing stuck stream`, or `Closing stuck EventSource connection` on
older builds), `wifidrop` (REST press, the client reconnects after 5 s idle like a browser `EventSource`) and `churn`
(40 connects, 20 of them reset, then a fresh client). `stream_runall.sh NAME PORT` uploads, captures and runs every
mode. Both builds should close the stalled stream at about 20 s and close the stream at the Wi-Fi drop.
- `sse_listing.py` prints stream 3's TCP connect and first event times. A late greeting (3 to 4 s) was TCP connect
time on the network (SYN retransmits), seen on the control too, not the device.
- On a noisy Wi-Fi network multi-client runs fell to 110 to 900 events per client with every client pausing together,
and a stalled client sometimes missed the EOF although the device logged `Closing stuck stream`. Alternate the
builds (flash A, 4 multi runs, flash B, repeat) before calling a rate difference.
- Log text check for refactors: collect every `[web_server*]` line from all captures, mask timestamps, source line
numbers and digits, `sort -u`, and diff the two builds' sets.
@@ -36,22 +36,3 @@ tracker as scan results: everything after the radio runs as normal.
first one it hears.
- Each frame arrives about 4 times; the counts matched the injection run exactly apart from one radio loss, so the
over the air comparison allows two missing publishes per tag.
## Every Xiaomi platform at once (#22727)
- `gen_all.py` writes `frames_all.hpp`, `xball_entities.yaml` and `expected_all.json`: one device per platform
(the 20 listener-only platforms, `xiaomi_mue4094rt`, `xiaomi_rtcgq02lm` hub, `xiaomi_body_scale`, ATC, PVVX),
10 cycles, every frame sent twice. `mibeacon.py` holds the frame builders and the C++ injector that `gen.py`
and `gen_all.py` share. Each entity logs `PUB <dev>.<key> <value>` on publish (binary sensors with
`trigger_on_initial_state: true`). Frame shapes that parse: long encrypted (fc 0x58) only up to 24 bytes, so
a 7 byte `th` object needs the short 19 byte frame, which only lywsd02mmc, xmwsdj04mmc, mjyd02yla, cgpr1 and
rtcgq02lm accept; hhccjcy10 is FD50 9 bytes and miscale v1 181D 10 bytes (no counter, the repeat publishes
again); mue4094rt is a plain fc 0x40 frame; body_scale an S200 0x4E16 object in a 24 byte short frame.
`xiaomi_mccgq02hl` publishes 9 of 10: the first frame is "closed", which equals its initial off state.
- `xball.yaml` is an ESP32-S3 (ESP-IDF, no network, USB_SERIAL_JTAG log) that injects one cycle per second from
20 s and logs free heap before and after (`IDLE` lines). `run_all.sh TREE NAME PORT` builds, flashes, captures
and counts; `count_all.py cap1 cap2 ...` also says whether the values are identical across builds.
- `xbsetup_priority.yaml`: `esphome config` with `setup_priority:` on a moved sensor, a moved binary sensor and
mue4094rt (still a component); remove one at a time, since validation stops at the first platform error.
- On #22727: 441 publishes on the PR, its stacked base and dev, identical values; dump text identical, the PR prints
the 20 listener dumps after `BLE Tracker:`.
@@ -21,39 +21,3 @@ uart_button/uartbtn*.yaml
value_range/rng*.yaml
ld2420_sim/ldenergy*.yaml
ld2420_sim/ldsimple*.yaml
apikey.txt
pio/
rp2_web_ble/*.log
rp2_web_ble/*.txt
rp2_web_ble/*.pcap
gatt_uuid/*.json
gatt_uuid/*.txt
web_auth/configs/*.log
web_auth/configs/*.txt
web_auth/configs/*.json
improv/*.log
npm1300/*-cap.txt
npm1300/*-cap.err
npm1300/*-run.log
rp2_sched_lock/*.txt
rp2_sched_lock/*.log
climate_ir_loop/*-*.txt
climate_ir_loop/*-frames.json
climate_ir_loop/whirlpool/w??base.yaml
climate_ir_loop/whirlpool/w??pr.yaml
tuya_sim/*.txt
tuya_sim/*.json
remote_base/pronto/*.txt
remote_base/pronto/gen-common.yaml
remote_base/dumpall/dumpers.h
remote_base/dumpall/db.txt
remote_base/dumpall/dp.txt
remote_base/dumpall/vb.txt
remote_base/dumpall/vp.txt
xiaomi_ble/*-cap.txt
xiaomi_ble/frames_all.hpp
xiaomi_ble/xball_entities.yaml
xiaomi_ble/expected_all.json
xiaomi_ble/run_*.yaml
ble_scan_complete/*.txt
ble_scan_complete/*.err
@@ -1,88 +0,0 @@
"""
Rapid native API connect/disconnect stress test (from esphome#14687 and esphome#14706).
usage: connstress.py HOST [ITERATIONS] [CONCURRENCY] (default 500 iterations, 4 at a time)
env: API_KEY (base64 noise key; unset, empty or - for plaintext), API_PORT (default 6053), API_PASSWORD
Each attempt connects with login, disconnects and counts a failure on any error or a 10 s timeout on either step.
Exits 1 when any attempt failed. Watch the serial log for crashes: the client only sees resets.
"""
import asyncio
import contextlib
import os
import sys
import time
from aioesphomeapi import APIClient
HOST = sys.argv[1] if len(sys.argv) > 1 else sys.exit(__doc__)
PORT = int(os.environ.get("API_PORT", "6053"))
PASSWORD = os.environ.get("API_PASSWORD", "")
NOISE_PSK = (
None if os.environ.get("API_KEY") in (None, "", "-") else os.environ["API_KEY"]
)
async def connect_disconnect(client_id: int, iteration: int) -> tuple[int, bool, str]:
cli = APIClient(HOST, PORT, PASSWORD, noise_psk=NOISE_PSK)
try:
await asyncio.wait_for(cli.connect(login=True), timeout=10)
# a disconnect that hangs on a wedged device is a failure too, not a stuck slot
await asyncio.wait_for(cli.disconnect(), timeout=10)
return iteration, True, ""
except Exception as e: # noqa: BLE001 every failure is a counted result
return (
iteration,
False,
f"client{client_id} iter{iteration}: {type(e).__name__}: {e}",
)
finally:
with contextlib.suppress(Exception):
await asyncio.wait_for(cli.disconnect(force=True), timeout=10)
async def main() -> None:
iterations = int(sys.argv[2]) if len(sys.argv) > 2 else 500
concurrency = int(sys.argv[3]) if len(sys.argv) > 3 else 4
print(f"Iterations: {iterations}, Concurrency: {concurrency}", flush=True)
success = fail = 0
errors: list[str] = []
start = time.monotonic()
sem = asyncio.Semaphore(concurrency)
async def run(client_id: int, iteration: int) -> tuple[int, bool, str]:
async with sem:
return await connect_disconnect(client_id, iteration)
tasks = [asyncio.create_task(run(i % concurrency, i)) for i in range(iterations)]
for coro in asyncio.as_completed(tasks):
_, ok, err = await coro
if ok:
success += 1
else:
fail += 1
errors.append(err)
total = success + fail
if total % 10 == 0 or not ok:
rate = total / max(time.monotonic() - start, 1e-6)
print(
f"[{total}/{iterations}] ok={success} fail={fail} ({rate:.1f}/s)",
flush=True,
)
if err:
print(f" ERROR: {err}", flush=True)
elapsed = time.monotonic() - start
print(f"\nDone in {elapsed:.1f}s")
print(f"Success: {success}, Failed: {fail}, Rate: {iterations / elapsed:.1f}/s")
if errors:
print("\nLast 10 errors:")
for e in errors[-10:]:
print(f" {e}")
print("RESULT", "PASS" if fail == 0 else "FAIL")
sys.exit(1 if fail else 0)
if __name__ == "__main__":
asyncio.run(main())
@@ -4,7 +4,6 @@ Import it with this directory on sys.path (the drivers add their parent director
"""
import asyncio
from collections.abc import Awaitable, Callable
import contextlib
import os
import re
@@ -28,32 +27,19 @@ FATAL = (
)
async def connect(
host: str,
tries: int = 30,
timeout: float = 10,
port: int = 6053,
on_stop: Callable[[bool], Awaitable[None]] | None = None,
) -> APIClient:
async def connect(host: str, tries: int = 30, timeout: float = 10) -> APIClient:
"""Connect and log in, retrying once a second; exits after TRIES failures with the last error.
API_KEY in the env selects Noise; unset, empty or - is plaintext. ON_STOP is called when the connection ends.
API_KEY in the env selects Noise; unset, empty or - is plaintext.
"""
key = os.environ.get("API_KEY")
last: BaseException | None = None
for _ in range(tries):
c = APIClient(
host, port, None, noise_psk=None if key in (None, "", "-") else key
host, 6053, None, noise_psk=None if key in (None, "", "-") else key
)
live = [False]
# a failed attempt's cleanup may call on_stop too; pass on only stops of the returned client
async def stop(expected: bool, live: list[bool] = live) -> None:
if live[0] and on_stop is not None:
await on_stop(expected)
try:
await asyncio.wait_for(c.connect(on_stop=stop, login=True), timeout)
await asyncio.wait_for(c.connect(login=True), timeout)
except FATAL as e:
sys.exit(f"no API connection to {host}: {type(e).__name__}: {e}")
except Exception as e: # noqa: BLE001
@@ -63,7 +49,6 @@ async def connect(
await c.disconnect(force=True)
await asyncio.sleep(1)
else:
live[0] = True
return c
sys.exit(
f"no API connection to {host} after {tries} tries: {type(last).__name__}: {last}"
@@ -73,80 +58,3 @@ async def connect(
def log_text(msg) -> str:
"""The message of a subscribe_logs callback as text, without ANSI colour codes."""
return ANSI.sub("", msg.message.decode(errors="replace"))
def stop_event() -> tuple[asyncio.Event, Callable[[bool], Awaitable[None]]]:
"""An Event and an on_stop callback for connect() that sets it when the connection ends."""
ev = asyncio.Event()
async def on_stop(expected: bool) -> None:
ev.set()
return ev, on_stop
async def ble_connect(
cli: APIClient,
addr: int,
on_lost: Callable[[int], None] | None = None,
advert_timeout: float = 60,
timeout: float = 30,
) -> tuple[bool, int, int, Callable[[], None]]:
"""Wait for an advert from ADDR, then connect through the proxy without a cache (like Home Assistant).
Returns (connected, mtu, error, cancel); a missing advert or a connect timeout gives (False, 0, -1, cancel), so
callers can retry. ON_LOST(error) is called when the link drops later.
"""
seen = asyncio.Event()
unsub = cli.subscribe_bluetooth_le_raw_advertisements(
lambda m: (
seen.set() if any(a.address == addr for a in m.advertisements) else None
)
)
try:
await asyncio.wait_for(seen.wait(), advert_timeout)
except TimeoutError:
return False, 0, -1, lambda: None
finally:
unsub()
first: asyncio.Queue = asyncio.Queue()
pending = True
def on_state(connected: bool, mtu: int, err: int) -> None:
nonlocal pending
if pending:
pending = False
first.put_nowait((connected, mtu, err))
elif not connected and on_lost is not None:
on_lost(err)
cancel = await cli.bluetooth_device_connect(
addr,
on_state,
timeout=timeout,
address_type=0,
feature_flags=0xFF,
has_cache=False,
)
try:
connected, mtu, err = await asyncio.wait_for(first.get(), timeout + 5)
except TimeoutError:
return False, 0, -1, cancel
return connected, mtu, err, cancel
async def enable_notify(
cli: APIClient, addr: int, ch, on_notify: Callable[[int, bytearray], None]
) -> bool:
"""Start notify on characteristic CH and write its first CCCD (2902) like Home Assistant.
Returns False when the characteristic lists no CCCD.
"""
await cli.bluetooth_gatt_start_notify(addr, ch.handle, on_notify)
cccd = next(
(d for d in ch.descriptors if d.uuid.lower().startswith("00002902")), None
)
if cccd is None:
return False
await cli.bluetooth_gatt_write_descriptor(addr, cccd.handle, b"\x01\x00")
return True
@@ -1,129 +0,0 @@
"""Home Assistant stand-in for a bluetooth_proxy scan soak (esphome#22770).
usage: ha.py HOST OUT SECONDS [--toggle SECS [--jitter SECS]] [--mode passive|active|none]
Subscribes to raw advertisements and the scanner state like Home Assistant, sets the scanner mode once after connect
(passive by default, as HA does for its proxies) and with --toggle flips active/passive every SECS seconds to stress
the stop/restart path. Writes one line per event to OUT: scanner state changes, a per-second advert count, mode
requests, connection drops. Reconnects after a drop (device reboot) and keeps going until SECONDS have passed.
API_KEY in the env selects Noise.
"""
import argparse
import asyncio
import pathlib
import random
import sys
import time
_here = pathlib.Path(__file__).resolve().parent
# apiconn.py: copied next to it, or the skill scripts dir
sys.path[:0] = [str(_here), str(_here.parent)]
from apiconn import FATAL, connect, stop_event # noqa: E402
def main() -> None:
ap = argparse.ArgumentParser()
ap.add_argument("host")
ap.add_argument("out")
ap.add_argument("seconds", type=float)
ap.add_argument("--toggle", type=float, default=0)
ap.add_argument(
"--jitter",
type=float,
default=0,
help="random +- seconds added to each toggle period",
)
ap.add_argument("--mode", default="passive", choices=["passive", "active", "none"])
a = ap.parse_args()
asyncio.run(run(a))
async def run(a: argparse.Namespace) -> None:
from aioesphomeapi import BluetoothScannerMode
end = time.monotonic() + a.seconds
f = pathlib.Path(a.out).open("a", encoding="utf-8", buffering=1) # noqa: SIM115
def w(s: str) -> None:
f.write(
f"{time.strftime('%H:%M:%S')}.{int(time.time() * 1000) % 1000:03d} {s}\n"
)
session = 0
connected = False
while time.monotonic() < end:
session += 1
stopped, on_stop = stop_event()
# a try takes up to 11 s (10 s timeout, 1 s pause): bound the tries by the soak time left instead of
# cancelling connect(), which could leave a half open client behind
tries = max(1, int((end - time.monotonic()) // 11))
try:
c = await connect(a.host, tries=tries, on_stop=on_stop)
except SystemExit as e:
w(f"CONNECT FAILED {e}")
# a wrong key or a plaintext/Noise mismatch never recovers: stop instead of soaking without a client
if isinstance(e.__context__, FATAL):
raise
await asyncio.sleep(5)
continue
w(f"CONNECTED session={session}")
connected = True
adv = [0]
macs: set[int] = set()
def on_raw(resp, adv=adv, macs=macs) -> None:
for x in resp.advertisements:
adv[0] += 1
macs.add(x.address)
def on_state(st) -> None:
w(f"SCANNER state={st.state.name} mode={st.mode.name}")
unsub_raw = c.subscribe_bluetooth_le_raw_advertisements(on_raw)
unsub_state = c.subscribe_bluetooth_scanner_state(on_state)
if a.mode != "none":
m = (
BluetoothScannerMode.PASSIVE
if a.mode == "passive"
else BluetoothScannerMode.ACTIVE
)
c.bluetooth_scanner_set_mode(m)
w(f"SET_MODE {a.mode}")
last_toggle = time.monotonic()
period = a.toggle
active = a.mode == "active"
last_report = time.monotonic()
while time.monotonic() < end and not stopped.is_set():
await asyncio.sleep(0.05)
now = time.monotonic()
if a.toggle and now - last_toggle >= period:
active = not active
c.bluetooth_scanner_set_mode(
BluetoothScannerMode.ACTIVE
if active
else BluetoothScannerMode.PASSIVE
)
w(f"SET_MODE {'active' if active else 'passive'}")
last_toggle = now
period = a.toggle + random.uniform(-a.jitter, a.jitter)
if now - last_report >= 10:
w(f"ADV {adv[0]} in {now - last_report:.1f}s devices={len(macs)}")
adv[0] = 0
macs.clear()
last_report = now
if stopped.is_set():
w("DISCONNECTED")
await asyncio.sleep(2)
continue
unsub_raw()
unsub_state()
await c.disconnect()
w("DONE")
if not connected:
sys.exit("no API connection during the whole soak")
if __name__ == "__main__":
main()
@@ -1,78 +0,0 @@
substitutions:
name: scan2610
duration: 5s
esphome:
name: ${name}
esp32:
board: esp32-c6-devkitc-1
variant: esp32c6
flash_size: 4MB
framework:
type: esp-idf
logger:
level: VERBOSE
hardware_uart: USB_SERIAL_JTAG
logs:
esp32_ble_tracker: VERBOSE
esp32_ble: DEBUG
bluetooth_proxy: DEBUG
api: DEBUG
api.connection: DEBUG
api.service: DEBUG
sensor: INFO
text_sensor: INFO
debug: INFO
wifi: DEBUG
scheduler: DEBUG
component: DEBUG
ota:
- platform: esphome
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
esp32_ble_tracker:
scan_parameters:
interval: 1100ms
window: 1100ms
active: true
duration: ${duration}
bluetooth_proxy:
active: true
sensor:
- platform: wifi_signal
name: WiFi signal
update_interval: 60s
- platform: uptime
name: Uptime
- platform: debug
free:
name: Free memory
block:
name: Largest free block
min_free:
name: Lowest free memory
loop_time:
name: Loop time
debug:
update_interval: 60s
text_sensor:
- platform: version
name: ESPHome version
- platform: debug
reset_reason:
name: Reset reason
api:
reboot_timeout: 0s
encryption:
key: !secret api_key
@@ -1,126 +0,0 @@
#pragma once
// esphome#22770 instrumentation: follows one scan's duration timer through Bluedroid.
// Link with -Wl,--wrap= for BTM_BleScan, btu_start_timer, btm_ble_timeout, osi_alarm_new, btc_transfer_context.
#include <cstdint>
#include <esp_gap_ble_api.h>
#include <esp_timer.h>
#include <freertos/FreeRTOS.h>
extern "C" {
struct scan_inst_t {
uint32_t ble_scan_start; // BTM_BleScan(TRUE) calls
uint32_t ble_scan_start_bad; // ... that did not return BTM_CMD_STARTED
uint32_t ble_scan_stop; // BTM_BleScan(FALSE) calls
uint32_t timer_start; // btu_start_timer(BTU_TTYPE_BLE_SCAN)
uint32_t alarm_fire; // esp_timer fired for the scan timer entry (btc thread)
uint32_t ble_timeout; // btm_ble_timeout for the scan timer entry (btu thread)
uint32_t inq_cmpl_post; // INQ_CMPL handed to btc_transfer_context
uint32_t inq_cmpl_post_fail; // ... that failed
uint8_t pid;
void *scan_tle;
int64_t last_timer_start_us;
int64_t last_fire_us;
int64_t last_timeout_us;
int64_t last_inq_cmpl_us;
};
// Written from the Bluedroid and timer tasks, read on the main loop: every access holds the lock, since the 64 bit
// timestamps can tear on the 32 bit chips. Read with scan_inst_snapshot() and log the copy.
static scan_inst_t g_scan_inst{};
static portMUX_TYPE g_scan_inst_mux = portMUX_INITIALIZER_UNLOCKED;
static scan_inst_t scan_inst_snapshot() {
portENTER_CRITICAL(&g_scan_inst_mux);
scan_inst_t copy = g_scan_inst;
portEXIT_CRITICAL(&g_scan_inst_mux);
return copy;
}
typedef void (*scan_inst_cb_t)(void *);
// the scan timer entry's original alarm callback (btu_general_alarm_cb)
static scan_inst_cb_t s_orig_cb = nullptr;
static void scan_inst_shim(void *data) {
int64_t now = esp_timer_get_time();
portENTER_CRITICAL(&g_scan_inst_mux);
g_scan_inst.alarm_fire++;
g_scan_inst.last_fire_us = now;
portEXIT_CRITICAL(&g_scan_inst_mux);
s_orig_cb(data);
}
uint8_t __real_BTM_BleScan(uint8_t start, uint32_t duration, void *a, void *b, void *c);
uint8_t __wrap_BTM_BleScan(uint8_t start, uint32_t duration, void *a, void *b, void *c) {
uint8_t r = __real_BTM_BleScan(start, duration, a, b, c);
portENTER_CRITICAL(&g_scan_inst_mux);
if (start) {
g_scan_inst.ble_scan_start++;
if (r != 1 /* BTM_CMD_STARTED */)
g_scan_inst.ble_scan_start_bad++;
} else {
g_scan_inst.ble_scan_stop++;
}
portEXIT_CRITICAL(&g_scan_inst_mux);
return r;
}
void __real_btu_start_timer(void *p_tle, uint16_t type, uint32_t timeout_sec);
void __wrap_btu_start_timer(void *p_tle, uint16_t type, uint32_t timeout_sec) {
if (type == 109 /* BTU_TTYPE_BLE_SCAN */) {
int64_t now = esp_timer_get_time();
portENTER_CRITICAL(&g_scan_inst_mux);
g_scan_inst.scan_tle = p_tle;
g_scan_inst.timer_start++;
g_scan_inst.last_timer_start_us = now;
portEXIT_CRITICAL(&g_scan_inst_mux);
}
__real_btu_start_timer(p_tle, type, timeout_sec);
}
void __real_btm_ble_timeout(void *p_tle);
void __wrap_btm_ble_timeout(void *p_tle) {
int64_t now = esp_timer_get_time();
portENTER_CRITICAL(&g_scan_inst_mux);
if (p_tle == g_scan_inst.scan_tle) {
g_scan_inst.ble_timeout++;
g_scan_inst.last_timeout_us = now;
}
portEXIT_CRITICAL(&g_scan_inst_mux);
__real_btm_ble_timeout(p_tle);
}
void *__real_osi_alarm_new(const char *name, scan_inst_cb_t cb, void *data, uint64_t expire);
void *__wrap_osi_alarm_new(const char *name, scan_inst_cb_t cb, void *data, uint64_t expire) {
// btu_start_timer's wrap records the scan entry before the real one creates its alarm. Shim every alarm made for
// it, since Bluedroid may free and recreate it; they all use the same callback (btu_general_alarm_cb).
portENTER_CRITICAL(&g_scan_inst_mux);
void *scan_tle = g_scan_inst.scan_tle;
portEXIT_CRITICAL(&g_scan_inst_mux);
if (data != nullptr && data == scan_tle) {
s_orig_cb = cb;
return __real_osi_alarm_new(name, scan_inst_shim, data, expire);
}
return __real_osi_alarm_new(name, cb, data, expire);
}
struct scan_inst_msg_t {
uint8_t sig, aid, pid, act;
};
int __real_btc_transfer_context(scan_inst_msg_t *msg, void *arg, int arg_len, void *cp, void *fr);
int __wrap_btc_transfer_context(scan_inst_msg_t *msg, void *arg, int arg_len, void *cp, void *fr) {
bool cmpl = msg != nullptr && arg != nullptr && msg->sig == 1 && msg->act == ESP_GAP_BLE_SCAN_RESULT_EVT &&
arg_len >= 4 && msg->pid == 5 /* BTC_PID_GAP_BLE */ &&
static_cast<esp_ble_gap_cb_param_t *>(arg)->scan_rst.search_evt == ESP_GAP_SEARCH_INQ_CMPL_EVT;
int r = __real_btc_transfer_context(msg, arg, arg_len, cp, fr);
if (cmpl) {
int64_t now = esp_timer_get_time();
portENTER_CRITICAL(&g_scan_inst_mux);
g_scan_inst.pid = msg->pid;
g_scan_inst.inq_cmpl_post++;
g_scan_inst.last_inq_cmpl_us = now;
if (r != 0)
g_scan_inst.inq_cmpl_post_fail++;
portEXIT_CRITICAL(&g_scan_inst_mux);
}
return r;
}
}
@@ -1,40 +0,0 @@
# scan.yaml plus the Bluedroid scan timer counters (scan_inst.hpp) logged every 30 s as `scaninst`
substitutions:
name: scaninst
packages:
base: !include scan.yaml
esphome:
includes:
- scan_inst.hpp
platformio_options:
build_flags:
- -Wl,--wrap=BTM_BleScan
- -Wl,--wrap=btu_start_timer
- -Wl,--wrap=btm_ble_timeout
- -Wl,--wrap=osi_alarm_new
- -Wl,--wrap=btc_transfer_context
esp32_ble_tracker:
on_scan_end:
- lambda: id(scan_ends)++;
globals:
- id: scan_ends
type: uint32_t
initial_value: '0'
interval:
- interval: 30s
then:
- lambda: |-
const scan_inst_t g = scan_inst_snapshot();
int64_t now = esp_timer_get_time();
ESP_LOGI("scaninst", "start=%u bad=%u stop=%u timer=%u fire=%u timeout=%u cmpl_post=%u cmpl_fail=%u pid=%u ends=%u "
"ago_ms timer=%lld fire=%lld timeout=%lld cmpl=%lld",
(unsigned) g.ble_scan_start, (unsigned) g.ble_scan_start_bad, (unsigned) g.ble_scan_stop,
(unsigned) g.timer_start, (unsigned) g.alarm_fire, (unsigned) g.ble_timeout,
(unsigned) g.inq_cmpl_post, (unsigned) g.inq_cmpl_post_fail, (unsigned) g.pid, (unsigned) id(scan_ends),
(now - g.last_timer_start_us) / 1000, (now - g.last_fire_us) / 1000,
(now - g.last_timeout_us) / 1000, (now - g.last_inq_cmpl_us) / 1000);
@@ -1,15 +0,0 @@
#!/bin/bash
# usage: soak.sh TAG HOST SECONDS [ha.py options] serial capture plus HA stand-in, both bounded
# env: PORT (serial port), PY, API_KEY (falls back to apikey.txt next to the script; unset or - for plaintext)
W=$(cd "$(dirname "$0")" && pwd); cd "$W" || exit 1; T=$1; H=$2; S=$3; shift 3
PY=${PY:-python3}
PORT=${PORT:?serial port}
[ -z "${API_KEY+x}" ] && [ -f apikey.txt ] && API_KEY=$(cat apikey.txt)
$PY "$(dirname "$W")/serial_capture.py" "$PORT" "$S" "$T.txt" > /dev/null 2>&1 &
CAP=$!
API_KEY=$API_KEY PYTHONUNBUFFERED=1 $PY ha.py "$H" "$T-ha.txt" "$S" "$@" > "$T-ha.err" 2>&1 &
HA=$!
wait $CAP; CRC=$?
wait $HA; HRC=$?
echo "$T done: capture rc=$CRC ha.py rc=$HRC (see $T-ha.err)"
[ $CRC -eq 0 ] && [ $HRC -eq 0 ]
@@ -1,11 +0,0 @@
#!/bin/bash
# usage: wait.sh CAPTURE SECONDS [FROM_LINE] block until an anomaly after FROM_LINE or SECONDS pass, then print status
C=$1; L=${3:-1}
P="Scan timeout exceeded|Scan never terminated|Dropped [0-9]+ BLE|Guru|Backtrace|Unexpected|cmpl_fail=[1-9]|bad=[1-9]"
timeout "$2" tail -n +"$L" -F "$C" 2> /dev/null | grep -a -m1 -q -E "$P"
tail -n +"$L" "$C" | awk -v p="$P" -v t="$(date +%T)" '
$0 ~ p && shown < 5 { print; shown++ }
/Starting scan/ { starts++ }
/Scan complete, set/ { completes++ }
END { printf "%s starts=%d completes=%d\n", t, starts, completes }'
grep -a scaninst "$C" | tail -1 | cut -c1-260
@@ -1,57 +0,0 @@
"""Count rounds, BTU_TASK crashes, reconnects without reboot and MTU 517 in a central capture.
analyze.py CAPTURE
A round counts from its "start" line (a crashing round never logs "fired"); rounds that log "not queued" or
"aborted" are taken back out and listed apart. Exits 1 when no round counts, since such a capture tested nothing.
"""
import pathlib
import re
import sys
lines = (
pathlib.Path(sys.argv[1]).read_text(encoding="utf-8", errors="replace").splitlines()
)
rounds = crashes = other_panics = boots = connects = mtu517 = mtu_other = 0
not_queued = aborted = skipped = 0
reconnect_ok = reconnect_reboot = 0
pending = False # a round fired and we wait for the next connect
for ln in lines:
if "Stack canary watchpoint triggered (BTU_TASK)" in ln:
crashes += 1
elif "Guru Meditation" in ln or "abort() was called" in ln:
other_panics += 1
if "rst:0x" in ln:
boots += 1
if pending:
reconnect_reboot += 1
pending = False
if "CLI connected" in ln:
connects += 1
if pending:
reconnect_ok += 1
pending = False
if re.search(r"ROUND \d+ start", ln):
rounds += 1
pending = True
if re.search(r"ROUND \d+ (not queued|aborted)", ln):
rounds -= 1
pending = False
not_queued += "not queued" in ln
aborted += "aborted" in ln
skipped += "ROUND skipped" in ln
if m := re.search(r"cfg_mtu status 0, mtu (\d+)", ln):
if m.group(1) == "517":
mtu517 += 1
else:
mtu_other += 1
print(
f"rounds={rounds} btu_canary_crashes={crashes} other_panics={other_panics} boots={boots}"
)
print(
f"connects={connects} reconnect_without_reboot={reconnect_ok} reconnect_after_reboot={reconnect_reboot}"
)
print(f"mtu517={mtu517} mtu_other={mtu_other}")
print(f"not_queued={not_queued} aborted={aborted} skipped={skipped}")
if rounds <= 0:
sys.exit("FAIL: no round reached the BTU queue")
@@ -1,75 +0,0 @@
# Central for the Bluedroid queued MTU recursion repro (esphome#17954). Every connection: wait 1.5 s, then
# suspend BTU_TASK, local disconnect, 4 reads + 1 MTU request (or execute write with exec_write: "true") in one
# batch, resume. ble_client reconnects.
esp32:
board: esp32dev
framework:
type: esp-idf
sdkconfig_options:
CONFIG_FREERTOS_WATCHPOINT_END_OF_STACK: y
logger:
level: DEBUG
esp32_ble_tracker:
scan_parameters:
active: false
globals:
- {id: rounds, type: int, initial_value: "0"}
- {id: chr_handle, type: uint16_t, initial_value: "0"}
ble_client:
- mac_address: ${peer_mac}
id: srv
on_connect:
- logger.log: {format: "CLI connected", level: INFO}
# Runs inside SEARCH_CMPL, before ble_client releases the GATT cache (it has no other nodes)
- lambda: |-
esp_bt_uuid_t uuid = esp32_ble_tracker::ESPBTUUID::from_raw("5ec10001-0000-1000-8000-00805f9b34fb").get_uuid();
esp_gattc_char_elem_t elem;
uint16_t count = 1;
id(chr_handle) = 0;
if (esp_ble_gattc_get_char_by_uuid(id(srv)->get_gattc_if(), id(srv)->get_conn_id(), 1, 0xFFFF, uuid, &elem,
&count) == ESP_GATT_OK && count > 0)
id(chr_handle) = elem.char_handle;
- delay: 1500ms
- lambda: |-
if (!id(srv)->connected()) return;
esp_gatt_if_t gattc_if = id(srv)->get_gattc_if();
uint16_t conn_id = id(srv)->get_conn_id();
uint16_t handle = id(chr_handle);
if (handle == 0) {
ESP_LOGW("btu", "ROUND skipped, no characteristic");
return;
}
int n = ++id(rounds);
ESP_LOGI("btu", "ROUND %d start conn_id=%u handle=0x%04x", n, conn_id, handle);
TaskHandle_t btu = xTaskGetHandle("BTU_TASK");
if (btu == nullptr) {
// vTaskSuspend(nullptr) would suspend this task forever
ESP_LOGE("btu", "ROUND %d aborted, no BTU_TASK", n);
return;
}
vTaskSuspend(btu);
// a rejected disconnect leaves the link up, so the queued calls never take the failing send path
esp_err_t err = esp_ble_gap_disconnect(id(srv)->get_remote_bda());
for (int i = 0; i < 4; i++) {
esp_err_t e = esp_ble_gattc_read_char(gattc_if, conn_id, handle, ESP_GATT_AUTH_REQ_NONE);
if (e != ESP_OK)
err = e;
}
esp_err_t e = ${exec_write} ? esp_ble_gattc_execute_write(gattc_if, conn_id, true)
: esp_ble_gattc_send_mtu_req(gattc_if, conn_id);
if (e != ESP_OK)
err = e;
delay(30);
vTaskResume(btu);
// a round whose calls were rejected never reached the BTU queue, so it tests nothing
if (err != ESP_OK) {
ESP_LOGW("btu", "ROUND %d not queued: %s", n, esp_err_to_name(err));
} else {
ESP_LOGI("btu", "ROUND %d fired", n);
}
on_disconnect:
- logger.log: {format: "CLI disconnected", level: INFO}
@@ -1,9 +0,0 @@
esphome:
name: btuclibase # one name per build (btuclibase, btuclipr)
substitutions:
peer_mac: "AA:BB:CC:DD:EE:FF" # the peer S3 Bluetooth MAC (Wi-Fi MAC + 2)
exec_write: "false"
packages:
common: !include btu-common.yaml
@@ -1,31 +0,0 @@
# Peer for the Bluedroid queued MTU recursion repro: a GATT server with one readable and writable characteristic
esphome:
name: btusrv
esp32:
variant: esp32s3
board: esp32-s3-devkitc-1
framework:
type: esp-idf
logger:
level: DEBUG
hardware_uart: USB_SERIAL_JTAG
esp32_ble:
name: btusrv
esp32_ble_server:
on_connect:
- logger.log: {format: "SRV connected", level: INFO}
on_disconnect:
- logger.log: {format: "SRV disconnected", level: INFO}
services:
- uuid: 5ec10000-0000-1000-8000-00805f9b34fb
advertise: true
characteristics:
- id: numc
uuid: 5ec10001-0000-1000-8000-00805f9b34fb
read: true
write: true
value: [42]
@@ -1,15 +0,0 @@
esphome:
name: btuxwbase # one name per build (btuxwbase, btuxwpr)
substitutions:
peer_mac: "AA:BB:CC:DD:EE:FF" # the peer S3 Bluetooth MAC (Wi-Fi MAC + 2)
exec_write: "true"
esp32:
framework:
sdkconfig_options:
# GATT API trace prints one "GATTC_ExecuteWrite conn_id=" line per execute send
CONFIG_BT_LOG_GATT_TRACE_LEVEL_API: y
packages:
common: !include btu-common.yaml
@@ -1,116 +0,0 @@
#ifndef FUZZ22773_CLIMATE_FUZZ_H
#define FUZZ22773_CLIMATE_FUZZ_H
// Test only (esphome#22773 A/B): feeds a climate's own frames, mutated copies and random frames to on_receive
// and prints whether each was accepted and the state that followed.
#include <cstdio>
#include <random>
#include <vector>
#include "esphome/components/climate_ir/climate_ir.h"
#include "esphome/components/remote_base/aeha_protocol.h"
#include "esphome/components/remote_base/remote_base.h"
#include "esphome/core/application.h"
#include "esphome/core/preferences.h"
namespace esphome::fuzz22773 {
class Cap : public remote_base::RemoteTransmitterBase {
public:
Cap() : RemoteTransmitterBase(nullptr) {}
std::vector<remote_base::RawTimings> frames;
protected:
void send_internal(uint32_t, uint32_t) override { this->frames.push_back(this->temp_.get_data()); }
};
template<typename C> void prep(C &c, Cap &tx) {
if (global_preferences != nullptr)
global_preferences->reset();
static char name[] = "fuzz";
static char fn[] = "";
App.pre_setup(name, sizeof(name) - 1, fn, 0);
c.set_transmitter(&tx);
static_cast<Component &>(c).setup();
}
template<typename C> void print_state(const char *tag, int i, bool ok, C &c) {
printf("FUZZ %s %d ok=%d mode=%d t=%.2f fan=%d swing=%d\n", tag, i, ok ? 1 : 0, (int) c.mode,
(double) c.target_temperature, c.fan_mode.has_value() ? (int) *c.fan_mode : -1, (int) c.swing_mode);
}
// A random AEHA frame of len bytes; half start with the climate's header (set_header), 3 in 4 use its address
template<typename Header>
void aeha_random(std::mt19937 &rng, remote_base::RawTimings &raw, size_t len, uint16_t address, Header set_header) {
uint8_t d[35];
for (auto &b : d)
b = rng();
if (rng() % 2)
set_header(d);
remote_base::RemoteTransmitData tx;
remote_base::AEHAProtocol().encode(&tx, (rng() % 4) ? address : uint16_t(rng()), d, len);
raw = tx.get_data();
}
// make() returns a fresh climate; random_frame(rng, raw) fills raw with a random frame of the protocol, without the
// trailing idle gap
template<typename Make, typename Rand> void run(const char *tag, Make make, Rand random_frame) {
std::mt19937 rng(22773);
// Valid frames: everything the climate itself sends
std::vector<remote_base::RawTimings> valid;
{
Cap tx;
auto c = make();
prep(*c, tx);
for (auto mode : {climate::CLIMATE_MODE_OFF, climate::CLIMATE_MODE_COOL, climate::CLIMATE_MODE_HEAT,
climate::CLIMATE_MODE_DRY, climate::CLIMATE_MODE_FAN_ONLY, climate::CLIMATE_MODE_HEAT_COOL})
for (float t : {16.0f, 18.0f, 21.0f, 24.0f, 27.0f, 30.0f, 60.0f, 66.0f, 72.0f, 80.0f, 88.0f})
for (auto fan : {climate::CLIMATE_FAN_AUTO, climate::CLIMATE_FAN_LOW, climate::CLIMATE_FAN_MEDIUM,
climate::CLIMATE_FAN_HIGH}) {
auto call = c->make_call();
call.set_mode(mode);
call.set_target_temperature(t);
call.set_fan_mode(fan);
call.perform();
}
valid = tx.frames;
for (auto &v : valid)
v.push_back(-20000); // the idle gap a receiver records after the last mark
}
int i = 0;
auto feed = [&](const remote_base::RawTimings &raw, const char *kind) {
Cap tx;
auto c = make();
prep(*c, tx);
remote_base::RemoteReceiveData data(raw, 25, remote_base::TOLERANCE_MODE_PERCENTAGE);
bool ok = static_cast<remote_base::RemoteReceiverListener &>(*c).on_receive(data);
printf("%s ", kind);
print_state(tag, i++, ok, *c);
};
for (const auto &v : valid) {
feed(v, "valid");
for (int m = 0; m < 4; m++) {
auto raw = v;
if (raw.size() < 8)
continue;
// a space swapped with another space's length flips a bit (or does nothing when they are equal)
size_t a = 3 + rng() % (raw.size() - 3), b = 3 + rng() % (raw.size() - 3);
if (raw[a] < 0 && raw[b] < 0)
std::swap(raw[a], raw[b]);
else
raw[a] = raw[a] < 0 ? -int32_t(100 + rng() % 3000) : int32_t(100 + rng() % 3000);
feed(raw, "mut");
}
auto t = v;
t.resize(rng() % t.size());
feed(t, "trunc");
}
for (int r = 0; r < 3000; r++) {
remote_base::RawTimings raw;
random_frame(rng, raw);
raw.push_back(-20000);
feed(raw, "rand");
}
fflush(stdout);
}
} // namespace esphome::fuzz22773
#endif
@@ -1,17 +0,0 @@
#include <gtest/gtest.h>
#include <memory>
#include "climate_fuzz.hpp"
#include "esphome/components/friedrich/friedrich.h"
namespace esphome::fuzz22773 {
TEST(Fuzz22773, Friedrich) {
run("friedrich", [] { return std::make_unique<friedrich::FriedrichClimate>(); },
[](std::mt19937 &rng, remote_base::RawTimings &raw) {
size_t len = (rng() % 3 == 0) ? 2 + rng() % 34 : ((rng() & 1) ? 14 : 5);
aeha_random(rng, raw, len, 0x28C6, [](uint8_t *d) {
d[0] = 0x00;
d[1] = 0x08;
d[2] = 0x08;
});
});
}
} // namespace esphome::fuzz22773
@@ -1,19 +0,0 @@
#include <gtest/gtest.h>
#include <memory>
#include "climate_fuzz.hpp"
#include "esphome/components/haier_tundra/haier_tundra.h"
#include "esphome/components/remote_base/haier_protocol.h"
namespace esphome::fuzz22773 {
TEST(Fuzz22773, HaierTundra) {
run("haier_tundra", [] { return std::make_unique<haier_tundra::HaierTundra>(haier_tundra::HAIER_HEADER_UNIT_A); },
[](std::mt19937 &rng, remote_base::RawTimings &raw) {
uint8_t d[16];
size_t len = (rng() % 3 == 0) ? 1 + rng() % 16 : 13;
for (auto &b : d) b = rng();
if (rng() % 2) d[0] = haier_tundra::HAIER_HEADER_UNIT_A;
remote_base::RemoteTransmitData tx;
remote_base::HaierProtocol().encode(&tx, d, len);
raw = tx.get_data();
});
}
} // namespace esphome::fuzz22773
@@ -1,12 +0,0 @@
from esphome.components import remote_base
from esphome.types import ConfigType
from tests.testing_helpers import ComponentManifestOverride
def override_manifest(manifest: ComponentManifestOverride) -> None:
manifest.auto_load = ["climate_ir", "climate"]
async def to_code_testing(config: ConfigType) -> None:
remote_base.request_protocol("haier")
manifest.to_code = to_code_testing
@@ -1,16 +0,0 @@
#include <gtest/gtest.h>
#include <memory>
#include "climate_fuzz.hpp"
#include "esphome/components/mistral_ir/mistral_ir.h"
namespace esphome::fuzz22773 {
TEST(Fuzz22773, MistralIR) {
run("mistral", [] { return std::make_unique<mistral_ir::MistralIR>(); },
[](std::mt19937 &rng, remote_base::RawTimings &raw) {
size_t len = (rng() % 3 == 0) ? 2 + rng() % 34 : 12;
aeha_random(rng, raw, len, mistral_ir::MISTRAL_ADDRESS, [](uint8_t *d) {
d[0] = 0x56;
d[2] = 0x08;
});
});
}
} // namespace esphome::fuzz22773
@@ -1,28 +0,0 @@
# Network, API and the VERBOSE logger setup shared by the climate loopback configs.
esphome:
name: ${name}
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
power_save_mode: none
logger:
level: VERBOSE
logs:
wifi: DEBUG
api: DEBUG
api.service: DEBUG
api.connection: DEBUG
scheduler: DEBUG
component: DEBUG
esp32.preferences: DEBUG
esp8266.preferences: DEBUG
preferences: DEBUG
mdns: DEBUG
sensor: DEBUG
remote_receiver: DEBUG
remote_transmitter: DEBUG
climate: DEBUG
api:
reboot_timeout: 0s
ota:
- platform: esphome
@@ -1,12 +0,0 @@
# Samsung, Friedrich, Siemens IRA211 and Mistral senders and receivers on one loopback pin.
packages:
base: !include base.yaml
climate:
- {platform: samsung, id: sam_tx, name: SAM TX, transmitter_id: irtx}
- {platform: samsung, id: sam_rx, name: SAM RX, transmitter_id: irtx, receiver_id: irrx}
- {platform: friedrich, id: fr_tx, name: FR TX, transmitter_id: irtx}
- {platform: friedrich, id: fr_rx, name: FR RX, transmitter_id: irtx, receiver_id: irrx}
- {platform: siemens_ira211, id: si_tx, name: SI TX, transmitter_id: irtx}
- {platform: siemens_ira211, id: si_rx, name: SI RX, transmitter_id: irtx, receiver_id: irrx}
- {platform: mistral_ir, id: mi_tx, name: MI TX, transmitter_id: irtx}
- {platform: mistral_ir, id: mi_rx, name: MI RX, transmitter_id: irtx, receiver_id: irrx}
@@ -1,5 +0,0 @@
substitutions:
name: c32pr
packages:
common: !include c-common.yaml
board: !include esp32-ir.yaml
@@ -1,98 +0,0 @@
"""compare.py A B [TOL] [--flat]: compare two runs (A-frames.json, B-frames.json from frames.py).
Per step (default): frames equal within TOL us and RX states equal.
--flat: the whole frame sequence in order, timing by timing, ignoring step boundaries (for the D1 mini, where a log
line can land in the next step); reports every item over TOL and the largest deviation elsewhere.
"""
import json
import pathlib
import sys
args = [a for a in sys.argv[1:] if a != "--flat"]
FLAT = "--flat" in sys.argv[1:]
TOL = int(args[2]) if len(args) > 2 else 0
def load(name):
steps = json.loads(pathlib.Path(f"{name}-frames.json").read_text(encoding="utf-8"))
# an empty run must not compare as equal to another empty run
if not steps or not any(s["frames"] for s in steps):
sys.exit(f"{name}-frames.json has no steps or no frames")
return steps
def same_timing(u, v):
return (u > 0) == (v > 0) and abs(u - v) <= TOL
def frame_dev(p, q):
"""Largest deviation of two frames, or None when they differ in length or a timing is over TOL."""
if len(p) != len(q) or not all(
same_timing(u, v) for u, v in zip(p, q, strict=False)
):
return None
return max((abs(u - v) for u, v in zip(p, q, strict=False)), default=0)
def flat(a, b):
fa = [f for s in a for f in s["frames"]]
fb = [f for s in b for f in s["frames"]]
mx = bad = items = 0
print("frames", len(fa), len(fb))
if len(fa) != len(fb):
print("DIFF frame counts differ (a lost or extra transmission)")
bad += 1
for i, (p, q) in enumerate(zip(fa, fb, strict=False)):
if len(p) != len(q):
print("frame", i, "length", len(p), len(q))
bad += 1
continue
for j, (u, v) in enumerate(zip(p, q, strict=False)):
items += 1
if same_timing(u, v):
mx = max(mx, abs(u - v))
else:
print("frame", i, "item", j, u, v)
bad += 1
print(f"items {items} over_tol {bad} max_dev_within_tol {mx}")
return bad == 0
def per_step(a, b):
if len(a) != len(b):
print(f"DIFF step counts {len(a)} vs {len(b)} (an interrupted run?)")
sys.exit(1)
same_f = same_s = maxdev = 0
for x, y in zip(a, b, strict=False):
assert x["step"] == y["step"]
fa, fb = x["frames"], y["frames"]
devs = [frame_dev(p, q) for p, q in zip(fa, fb, strict=False)]
ok = len(fa) == len(fb) and None not in devs
if ok:
maxdev = max([maxdev, *devs])
same_f += ok
st = x["rx"] == y["rx"]
same_s += st
if not ok or not st:
print(
"DIFF",
x["step"],
"frames",
len(fa),
len(fb),
[len(p) for p in fa],
[len(q) for q in fb],
)
for k in x["rx"]:
if x["rx"][k] != y["rx"].get(k):
print(" ", k, x["rx"][k], "|", y["rx"].get(k))
print(
f"steps {len(a)}/{len(b)} frames_equal {same_f} rx_states_equal {same_s} "
f"total_frames {sum(len(s['frames']) for s in a)}/{sum(len(s['frames']) for s in b)} max_dev_us {maxdev}"
)
return same_f == len(a) and same_s == len(a)
if not (flat if FLAT else per_step)(load(args[0]), load(args[1])):
sys.exit(1)
@@ -1,74 +0,0 @@
"""drive.py IP STEP...: STEP is 'Name:mode=cool,t=24,fan=low,swing=vertical' or 'svc:action'; prints climate states after each."""
import asyncio
import os
import pathlib
import sys
import time
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from aioesphomeapi import (
ClimateFanMode,
ClimateInfo,
ClimateMode,
ClimatePreset,
ClimateSwingMode,
)
from apiconn import connect
MODES = {m.name.lower(): m for m in ClimateMode}
FANS = {m.name.lower(): m for m in ClimateFanMode}
SWINGS = {m.name.lower(): m for m in ClimateSwingMode}
PRESETS = {m.name.lower(): m for m in ClimatePreset}
async def main() -> None:
c = await connect(sys.argv[1])
ents, svcs = await c.list_entities_services()
clim = {e.name: e for e in ents if isinstance(e, ClimateInfo)}
byk = {e.key: e.name for e in clim.values()}
st = {}
c.subscribe_states(
lambda s: st.__setitem__(byk[s.key], s) if s.key in byk else None
)
await asyncio.sleep(2)
def show(tag):
for n in sorted(st):
s = st[n]
print(
f" {tag} {n:11s} mode={s.mode.name} target={s.target_temperature:g} fan={s.fan_mode.name} swing={s.swing_mode.name} preset={s.preset.name}"
)
show("init")
for step in sys.argv[2:]:
name, _, args = step.partition(":")
print(time.strftime("%H:%M:%S"), "STEP", step, flush=True)
if name == "svc":
sname, _, sargs = args.partition("?")
data = {
k: v == "1"
for k, v in (a.split("=") for a in filter(None, sargs.split("&")))
}
await c.execute_service(next(s for s in svcs if s.name == sname), data)
else:
kw = {}
for a in filter(None, args.split(",")):
k, v = a.split("=")
if k == "mode":
kw["mode"] = MODES[v]
elif k == "t":
kw["target_temperature"] = float(v)
elif k == "fan":
kw["fan_mode"] = FANS[v]
elif k == "swing":
kw["swing_mode"] = SWINGS[v]
elif k == "preset":
kw["preset"] = PRESETS[v]
c.climate_command(clim[name].key, **kw)
await asyncio.sleep(float(os.environ.get("STEP_S", "2.5")))
show("after")
await c.disconnect()
asyncio.run(main())
@@ -1,24 +0,0 @@
# One-pin IR loopback on an ESP32 devkit: transmitter and receiver share GPIO27, RMT reads its own output.
# Override the substitutions (or `dump:`) from the including config.
substitutions:
pin: GPIO27
rmt_symbols: "384"
receive_symbols: "384"
esp32:
board: esp32dev
framework:
type: esp-idf
remote_transmitter:
id: irtx
pin:
number: ${pin}
allow_other_uses: true
carrier_duty_percent: 100%
remote_receiver:
id: irrx
pin:
number: ${pin}
allow_other_uses: true
dump: raw
rmt_symbols: ${rmt_symbols}
receive_symbols: ${receive_symbols}
@@ -1,23 +0,0 @@
# One-pin IR loopback on a D1 mini: transmitter and receiver share GPIO12. The receiver's setup leaves the shared pin
# an input, so an interval drives it as an output again. Override the substitutions (or `dump:`) from the including
# config.
substitutions:
pin: "12"
esp8266:
board: d1_mini
remote_transmitter:
id: irtx
pin:
number: ${pin}
allow_other_uses: true
carrier_duty_percent: 100%
remote_receiver:
id: irrx
pin:
number: ${pin}
allow_other_uses: true
dump: raw
interval:
- interval: 250ms
then:
- lambda: pinMode(${pin}, OUTPUT);
@@ -1,53 +0,0 @@
"""frames.py NAME: per step, the raw frames and the RX states. Prints JSON to NAME-frames.json."""
import json
import pathlib
import re
import sys
n = sys.argv[1]
steps = []
cur = None
with pathlib.Path(f"{n}-drive.txt").open(encoding="utf-8") as fh:
for text in fh:
m = re.match(r"(\d\d:\d\d:\d\d) STEP (.*)", text)
if m:
cur = {"t": m[1], "step": m[2], "rx": {}, "tx": {}, "frames": []}
steps.append(cur)
m = re.match(r"\s+after (\S+ \S+)\s+(.*)", text)
if m and cur:
(cur["rx"] if m[1].endswith("RX") else cur["tx"])[m[1]] = m[2]
frames = []
buf = None
with pathlib.Path(f"{n}-ser.txt").open(encoding="utf-8", errors="replace") as fh:
for text in fh:
m = re.match(r"\[(\d\d:\d\d:\d\d)\]\[I\]\[remote\.raw(?::\d+)?\]: ?(.*)", text)
if not m:
continue
ts, body = m[1], m[2]
if body.startswith("Received Raw:"):
buf = [ts, body[len("Received Raw:") :]]
elif buf is not None:
buf[1] += body
# the raw dumper splits long frames into lines that end with a comma; the last line does not
if buf is not None and not body.rstrip().endswith(","):
frames.append(
(
buf[0],
[
int(x)
for x in buf[1].replace(" ", "").strip(",").split(",")
if x
],
)
)
buf = None
for ts, f in frames:
owner = None
for s in steps:
if s["t"] <= ts:
owner = s
if owner:
owner["frames"].append(f)
pathlib.Path(f"{n}-frames.json").write_text(json.dumps(steps), encoding="utf-8")
print(n, "steps", len(steps), "frames", sum(len(s["frames"]) for s in steps))
@@ -1,30 +0,0 @@
#!/bin/bash
# run.sh NAME PORT [STEPS_FILE]: serial capture with a reset, then the prelude and steps of STEPS_FILE (default
# steps.sh here, which sets the PRE and STEPS arrays) through drive.py; writes NAME-ser.txt, NAME-pre.txt and
# NAME-drive.txt. env: PY, BAUD (default 115200; use 921600 for long climate frames), STEP_S (default 2.5, the wait
# after each step; the prelude always uses 2.5 s since it is not compared)
D=$(cd "$(dirname "$0")" && pwd); S=$(dirname "$D"); N=${1:?name}; P=${2:?port}; PY=${PY:-python3}
# shellcheck source=/dev/null
source "${3:-$D/steps.sh}"
cd "$D" || exit 1
pkill -f "[s]erial_capture.py $P "; sleep 1
# a stale capture would feed the IP lookup below
rm -f "$D/$N-ser.txt"
nohup $PY "$S/serial_capture.py" "$P" 1000 "$D/$N-ser.txt" --reset --baud "${BAUD:-115200}" > /dev/null 2>&1 &
IP=
for _ in $(seq 1 60); do
IP=$(LC_ALL=C grep -a -o -E "IP Address: [0-9.]+" "$N-ser.txt" 2>/dev/null | tail -1 | cut -d' ' -f3)
[ -n "$IP" ] && break; sleep 2
done
[ -z "$IP" ] && { echo "no IP in $N-ser.txt"; pkill -f "[s]erial_capture.py $P "; exit 1; }
sleep 3
STEP_S=2.5 $PY drive.py "$IP" "${PRE[@]}" > "$N-pre.txt" 2>&1; PRC=$?
STEP_S=${STEP_S:-2.5} $PY drive.py "$IP" "${STEPS[@]}" > "$N-drive.txt" 2>&1; DRC=$?
sleep 2; pkill -f "[s]erial_capture.py $P "; sleep 1
echo "$N steps=$(grep -c STEP "$N-drive.txt") raw=$(LC_ALL=C grep -a -c 'Received Raw' "$N-ser.txt")" \
"boots=$(LC_ALL=C grep -a -c 'Running through setup\|rst cause' "$N-ser.txt")"
# a failed drive leaves a partial run: fail so frames.py and compare.py are not run on it
if [ $PRC -ne 0 ] || [ $DRC -ne 0 ]; then
echo "$N drive failed: prelude rc=$PRC steps rc=$DRC (see $N-pre.txt, $N-drive.txt)"
exit 1
fi
@@ -1,14 +0,0 @@
# prelude puts every sender in the same state on both builds; frames before the first MARK step are not compared
PRE=("SAM TX:mode=off,t=24,fan=auto,swing=off" "FR TX:mode=off,t=72,fan=auto,swing=off" "SI TX:mode=off,t=21,fan=auto,preset=comfort" "MI TX:mode=off,t=24,fan=auto")
STEPS=()
for p in "SAM TX" "FR TX" "MI TX"; do
for m in heat_cool cool dry fan_only heat off; do STEPS+=("${p}:mode=$m"); done
STEPS+=("${p}:mode=cool")
if [ "$p" = "FR TX" ]; then fans=(high medium low quiet auto); else fans=(low medium high auto); fi
for f in "${fans[@]}"; do STEPS+=("${p}:fan=$f"); done
[ "$p" = "SAM TX" ] && for s in vertical horizontal both off; do STEPS+=("${p}:swing=$s"); done
STEPS+=("${p}:mode=off")
done
STEPS+=("SI TX:mode=heat_cool" "SI TX:preset=eco" "SI TX:preset=comfort")
for f in low medium high auto; do STEPS+=("SI TX:fan=$f"); done
STEPS+=("SI TX:mode=off")
@@ -1,52 +0,0 @@
# API actions that send whirlpool frames the sender cannot produce; add a board package and whirlpool/cl.yaml.
packages:
base: !include base.yaml
remote_receiver:
# a short frame ends in the idle gap, which the decoder takes as the second divider only up to 9,950 us
# (7,960 + 25%); the 9 ms header mark must stay below idle
idle: 9500us
api:
actions:
# A Hitachi frame the sender cannot produce: temperature nibble 15 (16 + 15 = 31 C, above the 30 C max), cool, fan low
- action: inject_t15
then:
- lambda: |-
static constexpr remote_base::BitTimings T{.bit_mark = 572, .one_space = 1659, .zero_space = 553};
uint8_t s[21] = {0};
s[0] = 0xA9; s[1] = 0x59; s[2] = 0x03; s[3] = 0xF2; s[6] = 0x80; s[10] = 0x20; s[15] = 0x02;
s[18] = 0x38; s[19] = 0x08;
s[13] = xor_bytes(&s[2], 11);
s[20] = xor_bytes(&s[14], 6);
ESP_LOGI("w22704", "inject t15");
auto call = id(irtx).transmit();
auto *d = call.get_data();
d->set_carrier_frequency(38000);
d->mark(9000); d->space(4494);
for (int i = 0; i < 21; i++) {
if (i == 6 || i == 14) { d->mark(572); d->space(7960); }
d->item_bytes<T>(s + i, 1);
}
d->mark(572);
call.perform();
# A 14-byte frame (what some whirlpool remotes send): bytes 0 to 13, the second divider, then silence
- action: inject_short
variables:
hitachi: bool
then:
- lambda: |-
static constexpr remote_base::BitTimings T{.bit_mark = 572, .one_space = 1659, .zero_space = 553};
uint8_t s[14] = {0};
s[0] = hitachi ? 0xA9 : 0x83; s[1] = hitachi ? 0x59 : 0x06; s[2] = 0x03; s[3] = 0xE2; s[6] = 0x80;
if (hitachi) s[10] = 0x20;
s[13] = xor_bytes(&s[2], 11);
ESP_LOGI("w22704", "inject short hitachi=%d", (int) hitachi);
auto call = id(irtx).transmit();
auto *d = call.get_data();
d->set_carrier_frequency(38000);
d->mark(9000); d->space(4494);
for (int i = 0; i < 14; i++) {
if (i == 6) { d->mark(572); d->space(7960); }
d->item_bytes<T>(s + i, 1);
}
d->mark(572);
call.perform();
@@ -1,29 +0,0 @@
#!/bin/bash
# all.sh PORT BASE_TREE PR_TREE: per whirlpool model (DG11J1-3A, HITACHI_HCRA31NEWH, HISENSE_ARG), build ../base.yaml
# plus cl.yaml plus the D1 mini one-pin loopback from the base tree, then the PR tree, flash, run the whirlpool steps and split the frames. Base
# runs first per model, so the board ends on a PR build. Compare the hex lines with `grep -a "Sending:\|Received:"`.
# env: PY, BUILD_LOCK (optional)
W=$(cd "$(dirname "$0")" && pwd); L=$(dirname "$W"); S=$(dirname "$L"); P=${1:?port}
export PIO_CORE=${PIO_CORE:-$W/pio}
FAILED=()
for mm in dg:DG11J1-3A hi:HITACHI_HCRA31NEWH hs:HISENSE_ARG; do
m=${mm%%:*}
for v in base pr; do
if [ $v = base ]; then T=${2:?base tree}; else T=${3:?pr tree}; fi
N=w$m$v
printf 'substitutions:\n name: %s\n model: %s\npackages:\n common: !include ../base.yaml\n model: !include cl.yaml\n board: !include ../esp8266-ir.yaml\n' \
"$N" "${mm#*:}" > "$W/$N.yaml"
echo "== $N"
# a skipped variant must not leave an older run's outputs behind for the comparison
rm -f "$L/$N-ser.txt" "$L/$N-pre.txt" "$L/$N-drive.txt" "$L/$N-frames.json"
TREE=$T bash "$S/pio_build.sh" compile "$N" "$W/$N.yaml" || { FAILED+=("$N:compile"); continue; }
TREE=$T bash "$S/pio_build.sh" upload "$N" "$W/$N.yaml" "$P" || { FAILED+=("$N:upload"); continue; }
RPY=${PY:-$T/venv/bin/python}
(cd "$L" && STEP_S=4 PY=$RPY bash run.sh "$N" "$P" "$W/steps.sh" && $RPY frames.py "$N") || FAILED+=("$N:run")
done
done
if [ ${#FAILED[@]} -gt 0 ]; then
echo "DONE with ${#FAILED[@]} failed variant(s): ${FAILED[*]}"
exit 1
fi
echo DONE
@@ -1,3 +0,0 @@
climate:
- {platform: whirlpool, id: w_tx, name: W TX, model: "${model}", transmitter_id: irtx}
- {platform: whirlpool, id: w_rx, name: W RX, model: "${model}", transmitter_id: irtx, receiver_id: irrx}
@@ -1,6 +0,0 @@
PRE=("W TX:mode=off,t=24,fan=auto,swing=off")
STEPS=()
for m in heat_cool heat cool dry fan_only off; do STEPS+=("W TX:mode=${m}"); done
STEPS+=("W TX:mode=cool")
for f in low medium high auto; do STEPS+=("W TX:fan=${f}"); done
STEPS+=("W TX:mode=off")
@@ -1,102 +0,0 @@
"""quiet_then_subscribe.py HOST PORT OUT QUIET_BOOTS: on the first boot drain any crash record (the first log
subscribe replays and clears it), then stay connected without a log subscription for QUIET_BOOTS reboots, then
subscribe on the next boot and record the log (including a crash record replayed on subscribe) until a line with
REBOOT. The drain connection is dropped and replaced by one without a subscription before boot 1 reboots, so a crash
that needs a log subscriber is not recorded as a quiet one; a reboot during the drain, or a reconnect slow enough to
hide one, fails the run instead of shifting the boot count. Exits 1 when that line never comes or a quiet boot does
not reboot within QUIET_REBOOT_WAIT seconds (default 900). Every lost connection counts as a reboot, so run it on a
stable link.
env: API_KEY (unset for a plaintext API), QUIET_REBOOT_WAIT."""
import asyncio
import contextlib
import os
import pathlib
import sys
import time
from aioesphomeapi import LogLevel
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from apiconn import connect, log_text, stop_event # noqa: E402
# how long a quiet boot may run before its reboot; QUIET_REBOOT_WAIT in the env overrides it
REBOOT_WAIT = float(os.environ.get("QUIET_REBOOT_WAIT", "900"))
# a reconnect slower than this may have waited out a reboot (connect() retries once a second)
RECONNECT_MAX_S = 5.0
async def main(host: str, port: int, outf: str, quiet: int) -> None:
with pathlib.Path(outf).open("a", encoding="utf-8") as f:
def w(t: str) -> None:
f.write(f"{time.strftime('%H:%M:%S')} {t}\n")
f.flush()
boots = 0
while True:
lost, on_stop = stop_event()
cli = await connect(host, tries=300, timeout=5, port=port, on_stop=on_stop)
boots += 1
info = await cli.device_info()
w(
f"=== CONNECTED boot {boots} name={info.name} quiet={1 < boots <= quiet + 1}"
)
if boots > quiet + 1:
done = asyncio.Event()
def on_log(m, done: asyncio.Event = done) -> None:
t = log_text(m)
w(t)
if "REBOOT" in t:
done.set()
cli.subscribe_logs(
on_log, log_level=LogLevel.LOG_LEVEL_DEBUG, dump_config=False
)
try:
await asyncio.wait_for(done.wait(), 120)
except TimeoutError:
w("=== no REBOOT within 120 s")
with contextlib.suppress(Exception):
await cli.disconnect()
if not done.is_set():
sys.exit("FAIL: no REBOOT line within 120 s of subscribing")
return
if boots == 1:
# drain any crash record left by the reboot that brought this build up
cli.subscribe_logs(
lambda m: w("drain " + log_text(m)),
log_level=LogLevel.LOG_LEVEL_DEBUG,
dump_config=False,
)
await asyncio.sleep(3)
if lost.is_set():
w("=== boot 1 rebooted during the drain")
sys.exit("FAIL: boot 1 rebooted with the drain subscriber attached")
with contextlib.suppress(Exception):
await cli.disconnect()
lost, on_stop = stop_event()
t = time.monotonic()
cli = await connect(
host, tries=300, timeout=5, port=port, on_stop=on_stop
)
if (dt := time.monotonic() - t) > RECONNECT_MAX_S:
w(f"=== boot 1 reconnect took {dt:.1f} s")
sys.exit(
f"FAIL: the reconnect after the drain took {dt:.1f} s, boot 1 may have rebooted"
)
w("=== boot 1 reconnected without a log subscription")
# quiet: wait for the device to drop the connection (its reboot)
try:
await asyncio.wait_for(lost.wait(), REBOOT_WAIT)
except TimeoutError:
w(f"=== boot {boots}: no reboot within {REBOOT_WAIT:.0f} s")
sys.exit(
f"FAIL: boot {boots} did not reboot within {REBOOT_WAIT:.0f} s"
)
w("=== connection lost")
await asyncio.sleep(2)
asyncio.run(main(sys.argv[1], int(sys.argv[2]), sys.argv[3], int(sys.argv[4])))
@@ -1,62 +0,0 @@
"""cmp.py REF.json OTHER.json : list each UUID (service / characteristic / descriptor, in order) where OTHER differs
from REF, whether reads match (static characteristics), whether every notifying characteristic notified and
whether OTHER lost its link before the final disconnect. Exits 1 on any difference."""
import json
import pathlib
import sys
def flat(p):
d = json.loads(pathlib.Path(p).read_text(encoding="utf-8"))
rows = []
for s in d["services"]:
rows.append(("svc", s["uuid"].lower()))
for c in s["characteristics"]:
rows.append(("chr", c["uuid"].lower()))
rows += [("dsc", x.lower()) for x in c["descriptors"]]
return d, rows
rd, ref = flat(sys.argv[1])
od, oth = flat(sys.argv[2])
diff = [(a, b) for a, b in zip(ref, oth, strict=False) if a != b]
print(
f"entries ref={len(ref)} other={len(oth)} differing={len(diff) + abs(len(ref) - len(oth))}"
)
for a, b in diff:
print(f" {a[0]} ref {a[1]} other {b[1]}")
def chars(d: dict) -> list[dict]:
return [c for s in d["services"] for c in s["characteristics"]]
# compare by position: a UUID form bug renames the keys, so values and notify counts are matched by table order
problems = []
rc, oc = chars(rd), chars(od)
for i, (a, b) in enumerate(zip(rc, oc, strict=False)):
if a["props"] != b["props"]:
problems.append(
f"props {i} {b['uuid']} 0x{b['props']:02X} != ref 0x{a['props']:02X}"
)
rv, ov = rd["reads"].get(a["uuid"]), od["reads"].get(b["uuid"])
if str(ov).startswith("FAILED"):
problems.append(f"read {i} {b['uuid']} {ov}")
elif not a["props"] & 0x10 and rv is not None and ov != rv:
problems.append(f"read {i} {b['uuid']} {ov!r} != ref {rv!r}")
if a["props"] & 0x10:
n = od["notify"].get(b["uuid"])
if not isinstance(n, int) or n <= 0:
problems.append(f"notify {i} {b['uuid']} {n}")
# a link that dropped after the first notification still has counts, so the drop itself fails the run
if od.get("lost"):
problems.append(f"link lost before the disconnect: {od['lost']}")
print(f"notify {od['notify']}")
for x in problems:
print(" ", x)
if diff or len(ref) != len(oth) or problems:
sys.exit(
f"FAIL: {len(diff)} differing entries, {len(problems)} read or notify problems"
)
print("PASS")
@@ -1,100 +0,0 @@
"""List every service, characteristic and descriptor UUID a bluetooth_proxy reports for a peer, read each readable
characteristic, enable notify (CCCD write) on each notifying one and count 10 s of notifications, then disconnect.
A link lost before that disconnect is recorded as "lost" in the JSON, which cmp.py fails on.
usage: gattlist.py PROXY PEER_MAC OUT_JSON env: API_KEY (unset for a plaintext API). Retries 0x3E connect failures."""
import asyncio
import json
import pathlib
import sys
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from apiconn import ble_connect, connect, enable_notify # noqa: E402
async def once(host: str, addr: int) -> dict | None:
cli = await connect(host)
lost: list[int] = []
closing = False
def on_lost(e: int) -> None:
if not closing:
lost.append(e)
print(f"LINK LOST err={e}", flush=True)
c, _mtu, err, cancel = await ble_connect(cli, addr, on_lost)
if not c:
print(f"connect err={err}", flush=True)
cancel()
await cli.disconnect()
return None
svcs = await cli.bluetooth_gatt_get_services(addr)
out = {"services": [], "reads": {}, "notify": {}}
notif: dict[str, int] = {}
for s in svcs.services:
so = {"uuid": s.uuid, "characteristics": []}
for ch in s.characteristics:
so["characteristics"].append(
{
"uuid": ch.uuid,
"props": ch.properties,
"descriptors": [d.uuid for d in ch.descriptors],
}
)
if ch.properties & 0x02:
try:
out["reads"][ch.uuid] = bytes(
await cli.bluetooth_gatt_read(addr, ch.handle)
).hex()
except Exception as e: # noqa: BLE001
out["reads"][ch.uuid] = f"FAILED {type(e).__name__}"
if ch.properties & 0x10:
notif[ch.uuid] = 0
try:
if not await enable_notify(
cli,
addr,
ch,
lambda h, d, u=ch.uuid: (
notif.__setitem__(u, notif[u] + 1)
if isinstance(notif[u], int)
else None
),
):
notif[ch.uuid] = "FAILED no CCCD"
except Exception as e: # noqa: BLE001
notif[ch.uuid] = f"FAILED {type(e).__name__}"
out["services"].append(so)
await asyncio.sleep(10)
out["notify"] = notif
out["lost"] = lost
closing = True
await cli.bluetooth_device_disconnect(addr)
await asyncio.sleep(2)
cancel()
await cli.disconnect()
return out
async def main(host: str, peer: str, path: str) -> None:
addr = int(peer.replace(":", ""), 16)
for _attempt in range(6):
res = await once(host, addr)
if res:
break
await asyncio.sleep(10)
else:
sys.exit("FAIL: no connection")
pathlib.Path(path).write_text(json.dumps(res, indent=1), encoding="utf-8")
for s in res["services"]:
print(f"SVC {s['uuid']}")
for c in s["characteristics"]:
print(
f" CHR {c['uuid']} props=0x{c['props']:02X} read={res['reads'].get(c['uuid'], '-')}"
f" notify={res['notify'].get(c['uuid'], '-')}"
)
for d in c["descriptors"]:
print(f" DSC {d}")
asyncio.run(main(sys.argv[1], sys.argv[2], sys.argv[3]))
@@ -1,14 +0,0 @@
#!/bin/zsh
# run.sh NAME: build ../rp2_web_ble/picow.yaml as NAME, USB flash the Pico W proxy, list the GATT UUID server
# (server.yaml here) through it and compare with the ESP32 reference. Exit 0 only when the lists match.
# env: TREE, PY, PICO_SN, PEER_MAC (the server); esp32ref.json (gattlist.py through an ESP32 proxy) and apikey.txt here
D=${0:A:h}; N=$1; S=$D/..; T=${TREE:?set TREE}; PY=${PY:-$T/venv/bin/python}; C=$S/rp2_web_ble/picow.yaml
$S/pio_build.sh compile $N $C || exit 1
PORT=$($PY $S/ports.py --wait 2e8a --serial ${PICO_SN:?set PICO_SN} --timeout 30) || { echo "no pico"; exit 1; }
$S/pio_build.sh upload $N $C $PORT || exit 1
H=$($S/resolve.sh $N) || exit 1; sleep 5
rm -f $D/$N.json
API_KEY=$(cat $D/apikey.txt) timeout 600 $PY -I $D/gattlist.py $H ${PEER_MAC:?set PEER_MAC} $D/$N.json > $D/$N-list.txt
RC=$?; grep -v "^ *CHR 00002a\|DSC 00002902\|SVC 000018" $D/$N-list.txt
[ $RC = 0 ] && [ -s $D/$N.json ] || { echo "FAIL: gattlist.py exited $RC"; exit 1; }
$PY $D/cmp.py $D/esp32ref.json $D/$N.json
@@ -1,27 +0,0 @@
# GATT server (ESP32-S3) for the UUID form check: the ble_notify server (16 bit and 32-bit base UUIDs, notify) plus a
# 128 bit service whose characteristics and descriptors use 128 bit, 32-bit base (zero and nonzero low half) and
# 16 bit UUIDs, with a 2901 user description. Set the name with -s name.
packages:
notify: !include ../ble_notify/server.yaml
esp32_ble_server:
services:
- uuid: 0f1e2d3c-4b5a-6978-8796-a5b4c3d2e1f0
characteristics:
- uuid: 0f1e2d3c-4b5a-6978-8796-a5b4c3d2e1f1
read: true
value: "c128"
description: "text value"
descriptors:
- uuid: 5ec1ab01-0000-1000-8000-00805f9b34fb
value: "d32"
- uuid: 7d3e0a51-9c1b-4f2e-8a6d-1c2b3a4d5e6f
value: "d128"
- uuid: 5ec20000-0000-1000-8000-00805f9b34fb
read: true
value: "c32zero"
- uuid: 5ec2abcd-0000-1000-8000-00805f9b34fb
read: true
value: "c32"
- uuid: 2a19
read: true
value: [77]
@@ -99,9 +99,6 @@ class Client(BaseClient):
for line in out["STATE"]:
name, _, value = line.partition(" = ")
self.last[name] = value
# SUB lines are printed sorted below; SUBORDER keeps the order the device sent them
for i, line in enumerate(out["SUB"]):
print(f" {label} SUBORDER {i:03d} {line}")
for kind in ("SUB", "STATE", "ACTION"):
for line in sorted(out[kind]) if kind == "SUB" else out[kind]:
print(f" {label} {kind} {line}")
@@ -1,18 +0,0 @@
# Package for Improv tests: Wi-Fi with no network saved, web_server and improv_serial with a next_url, and an ALIVE
# line every 5 s. Substitution: name. Drop web_server on a LibreTiny board with no ap: (it boot-loops, see
# reference/recipes/improv.md).
esphome:
name: ${name}
wifi:
power_save_mode: none
web_server:
improv_serial:
next_url: https://example.com/?device_name={{device_name}}&ip_address={{ip_address}}
interval:
- interval: 5s
then:
- logger.log: "ALIVE"
@@ -1,15 +0,0 @@
# Classic ESP32 devkit with Improv over serial (UART0, shared with the log) and BLE. Copy it per build with its own
# name; run.sh wants the file name to match name.
substitutions:
name: improv32
packages:
common: !include common.yaml
esp32:
board: esp32dev
framework:
type: esp-idf
logger:
hardware_uart: UART0
improv_ble:
authorizer: none
next_url: https://example.com/?device_name={{device_name}}&ip_address={{ip_address}}
@@ -1,96 +0,0 @@
"""Improv over BLE: find the device by name, send WIFI_SETTINGS and print state, errors and the RPC result URLs.
usage: improvble.py NAME SECRETS (needs bleak; the device needs improv_ble authorizer: none and a next_url)"""
import asyncio
import pathlib
import sys
from bleak import BleakClient, BleakScanner
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from improvcli import load_wifi_secrets, strings # noqa: E402
BASE = "00467768-6228-2272-4663-2774782680"
STATE, ERROR, RPC, RESULT = BASE + "01", BASE + "02", BASE + "03", BASE + "04"
STATES = {1: "awaiting_auth", 2: "authorized", 3: "provisioning", 4: "provisioned"}
def pkt(cmd: int, data: bytes) -> bytes:
b = bytes([cmd, len(data)]) + data
return b + bytes([sum(b) & 0xFF])
async def main(name: str, secrets: str) -> None:
ssid, pw = (x.encode() for x in load_wifi_secrets(secrets))
dev = await BleakScanner.find_device_by_name(name, timeout=30)
if dev is None:
sys.exit(
"FAIL: device not found (a provisioned board stops advertising Improv)"
)
loop = asyncio.get_running_loop()
t0 = loop.time()
done = asyncio.Event()
errors: list = []
states: list[int] = []
async with BleakClient(dev) as c:
def on_state(_, d: bytearray) -> None:
if not d:
errors.append("empty state")
return
states.append(d[0])
print(f"{loop.time() - t0:6.2f} STATE {STATES.get(d[0], d[0])}", flush=True)
def on_error(_, d: bytearray) -> None:
if not d:
errors.append("empty error")
elif d[0]:
errors.append(d[0])
print(f"{loop.time() - t0:6.2f} ERROR {d[0]}", flush=True)
def on_result(_, d: bytearray) -> None:
if len(d) < 3 or d[1] != len(d) - 3 or sum(d[:-1]) & 0xFF != d[-1]:
errors.append("bad result frame")
print(
f"{loop.time() - t0:6.2f} BAD RESULT {bytes(d).hex()}", flush=True
)
return
try:
strs = strings(bytes(d[2:-1]))
except ValueError:
errors.append("string overruns the result")
strs = []
print(f"{loop.time() - t0:6.2f} RESULT cmd={d[0]} {strs}", flush=True)
if (
d[0] == 1
): # the WIFI_SETTINGS reply; the example sets next_url, so it carries a URL
if not any(x.startswith("http") for x in strs):
errors.append("no URL in result")
done.set()
await c.start_notify(STATE, on_state)
await c.start_notify(ERROR, on_error)
await c.start_notify(RESULT, on_result)
s = await c.read_gatt_char(STATE)
print(f"initial state {STATES.get(s[0], s[0]) if s else 'empty'}")
if not s or s[0] == 4:
sys.exit(
"FAIL: board already provisioned or no state; erase the flash first"
)
payload = bytes([len(ssid)]) + ssid + bytes([len(pw)]) + pw
await c.write_gatt_char(RPC, pkt(1, payload), response=True)
try:
await asyncio.wait_for(done.wait(), 45)
except TimeoutError:
print("no result within 45 s")
await asyncio.sleep(1)
last = await c.read_gatt_char(STATE)
states.append(last[0] if last else 0)
if not done.is_set() or errors or states[-1] != 4:
sys.exit(
f"FAIL (errors {errors}, last state {STATES.get(states[-1], states[-1])})"
)
print("PASS")
asyncio.run(main(sys.argv[1], sys.argv[2]))
@@ -1,339 +0,0 @@
"""Improv serial client that reads frames out of the log stream on the same port.
usage: improvcli.py PORT OUT --secrets FILE [--baud N] [--keep-dtr] [--settle S] [--skip-bad] [--allow-reset]
Runs: 300 stray bytes, state, info, network state, scan, provision with a bad password, provision with the real one,
state and network state again. Prints parsed frames; the raw stream overwrites OUT.
Exit 2 when the port does not open or the secrets file lacks a key. Exit 1 when any request goes unanswered, an
unexpected error or a malformed frame arrives, or the port resets or the log shows a reboot (unless --allow-reset).
Credentials come from the secrets file (wifi_ssid, wifi_password) and are never printed; the SSID is masked in scan
results."""
import argparse
import pathlib
import random
import sys
import time
import serial
import yaml
STATES = {
0: "stopped",
1: "awaiting_auth",
2: "authorized",
3: "provisioning",
4: "provisioned",
}
ERRORS = {
0: "none",
1: "invalid_rpc",
2: "unknown_rpc",
3: "unable_to_connect",
4: "not_authorized",
0xFF: "unknown",
}
def frame(cmd: int, payload: bytes = b"") -> bytes:
data = bytes([cmd, len(payload)]) + payload
body = b"IMPROV" + bytes([1, 3, len(data)]) + data
return body + bytes([sum(body) & 0xFF])
BOOT_MARKER = (
b"Running through setup()" # ESPHome logs it once per boot on every platform
)
def rpc_strings(data: bytes) -> bytes:
return data[2 : 2 + data[1]]
def strings(p: bytes) -> list[str]:
"""Split length-prefixed strings; raise ValueError when a prefix runs past the end."""
out = []
while p:
if 1 + p[0] > len(p):
raise ValueError("string overruns the payload")
out.append(p[1 : 1 + p[0]].decode(errors="replace"))
p = p[1 + p[0] :]
return out
def load_wifi_secrets(path: str) -> tuple[str, str]:
"""wifi_ssid and wifi_password from a secrets file; exits 2 (setup error) when it cannot be used."""
try:
with pathlib.Path(path).open(encoding="utf-8") as f:
sec = yaml.safe_load(f)
except (OSError, yaml.YAMLError) as exc:
print(f"cannot read secrets file: {exc}", file=sys.stderr)
sys.exit(2)
try:
raw = (sec["wifi_ssid"], sec["wifi_password"])
except (KeyError, TypeError) as exc:
print(f"secrets file lacks {exc}", file=sys.stderr)
sys.exit(2)
if not all(
isinstance(x, (str, int, float)) and not isinstance(x, bool) for x in raw
):
print("wifi_ssid and wifi_password must be plain values", file=sys.stderr)
sys.exit(2)
return str(raw[0]), str(raw[1]) # YAML reads an unquoted number as int
class Dev:
def __init__(
self, port: str, baud: int, out: str, keep_dtr: bool, ssid: str
) -> None:
self.ser = serial.Serial()
self.ser.port, self.ser.baudrate, self.ser.timeout = port, baud, 0.05
if not keep_dtr:
self.ser.dtr = False
self.ser.rts = False
self.reopen(setup=True)
self.buf = bytearray()
try:
self.log = pathlib.Path(out).open("wb") # noqa: SIM115
except OSError as exc:
print(f"cannot open output file: {exc}", file=sys.stderr)
sys.exit(2)
self.t0 = time.time()
self.events: list[tuple[float, int, bytes, bool]] = []
self.loglines = 0
self.reopens = 0
self.boots = 0
self.tail = b""
self.ssid = ssid
def reopen(self, setup: bool = False) -> None:
# Native USB ports re-enumerate on reset; wait up to 10 s for them.
err: Exception | None = None
for _ in range(40):
try:
if self.ser.is_open:
self.ser.close()
self.ser.open()
return
except (serial.SerialException, OSError) as exc:
err = exc
time.sleep(0.25)
print(
f"port did not {'open' if setup else 'come back'}: {err}", file=sys.stderr
)
sys.exit(2 if setup else 1)
def pump(self, dur: float) -> None:
end = time.time() + dur
while time.time() < end:
try:
d = self.ser.read(4096)
except serial.SerialException:
# A dropped native USB port usually means the board reset; make that visible.
self.reopens += 1
print(
f"{time.time() - self.t0:7.2f} PORT RESET (reopen {self.reopens})",
flush=True,
)
self.reopen()
continue
if d:
seen = self.tail + d
self.boots += seen.count(BOOT_MARKER)
self.tail = seen[-(len(BOOT_MARKER) - 1) :]
self.log.write(d)
self.log.flush()
self.buf += d
self.parse()
def parse(self) -> None:
while True:
i = self.buf.find(b"IMPROV")
if i < 0:
keep = max(0, len(self.buf) - 6)
self.loglines += self.buf[:keep].count(b"\n")
del self.buf[:keep]
return
self.loglines += self.buf[:i].count(b"\n")
del self.buf[:i]
if len(self.buf) < 9:
return
if self.buf[6] != 1: # not a version 1 header: "IMPROV" in log text
del self.buf[:6]
continue
ln = self.buf[8]
if len(self.buf) < 10 + ln:
return
body, ck = bytes(self.buf[: 9 + ln]), self.buf[9 + ln]
del self.buf[: 10 + ln]
typ, data = body[7], body[9:]
ok = (sum(body) & 0xFF) == ck
# A malformed frame is counted as bad, so the predicates never index past it.
short = len(data) < {1: 1, 2: 1, 4: 2}.get(typ, 0)
if typ == 4 and not short:
try:
# ESPHome sends one trailing byte after the strings (an unused checksum slot).
short = data[1] > len(data) - 2
strings(rpc_strings(data))
except ValueError:
short = True
self.events.append((time.time() - self.t0, typ, data, ok and not short))
self.show(typ, data, ok, short)
def show(self, typ: int, data: bytes, ok: bool, short: bool) -> None:
t = f"{time.time() - self.t0:7.2f}"
tag = "" if ok else " BADCHECKSUM"
if short:
print(f"{t} TYPE{typ} {data.hex()} MALFORMED{tag}")
elif typ == 1:
print(f"{t} STATE {STATES.get(data[0], data[0])}{tag}")
elif typ == 2:
print(f"{t} ERROR {ERRORS.get(data[0], data[0])}{tag}")
elif typ == 4:
strs = [
"<ssid>" if x == self.ssid else x for x in strings(rpc_strings(data))
]
print(f"{t} RPC cmd={data[0]} len={data[1]} {strs}{tag}")
else:
print(f"{t} TYPE{typ} {data.hex()}{tag}")
def send(self, cmd: int, payload: bytes = b"", wait: float = 2.0) -> None:
self.ser.write(frame(cmd, payload))
self.ser.flush()
self.pump(wait)
def wait_for(self, pred, timeout: float, start: int) -> tuple | None:
end = time.time() + timeout
while time.time() < end:
self.pump(0.2)
for e in self.events[start:]:
if pred(e):
return e
return None
def ask(
d: Dev, label: str, cmd: int, pred, timeout: float = 1.5, payload: bytes = b""
) -> bool:
"""Send one request and require a reply matching pred; any other nonzero ERROR frame fails it."""
print(f"== {label}")
start = len(d.events) # before sending, so an immediate reply counts
t = time.time()
d.send(cmd, payload, wait=0)
e = d.wait_for(pred, timeout, start)
print(
f"reply after {time.time() - t:.1f}s"
if e
else f"NO reply within {timeout:.0f}s"
)
errs = [
x for x in d.events[start:] if x[1] == 2 and x[3] and x[2][0] and not pred(x)
]
if errs:
print(f"unexpected errors: {[ERRORS.get(x[2][0], x[2][0]) for x in errs]}")
return e is not None and not errs
def rpc(cmd: int):
return lambda e: e[1] == 4 and e[3] and e[2][0] == cmd
def state(e: tuple) -> bool:
return e[1] == 1 and e[3]
def main() -> None:
a = argparse.ArgumentParser()
a.add_argument("port")
a.add_argument("out")
a.add_argument("--baud", type=int, default=115200)
a.add_argument(
"--keep-dtr",
action="store_true",
help="leave DTR asserted (Pico 2 W CDC sends nothing without)",
)
a.add_argument("--secrets", required=True)
a.add_argument("--settle", type=float, default=3)
a.add_argument("--skip-bad", action="store_true")
a.add_argument(
"--allow-reset",
action="store_true",
help="a port reset or a reboot in the log does not fail the run",
)
o = a.parse_args()
ssid, pw = load_wifi_secrets(o.secrets)
s, w = ssid.encode(), pw.encode()
print(f"password length {len(w)} bytes")
d = Dev(o.port, o.baud, o.out, o.keep_dtr, ssid)
d.pump(o.settle)
boots_at_start = d.boots
before = d.loglines
print("== stray bytes")
j = len(d.events)
junk = bytes(
random.choice([b for b in range(256) if b != 0x49]) for _ in range(300)
)
d.ser.write(junk)
d.ser.flush()
d.pump(1.0)
# Stray bytes must be ignored silently; the state reply below shows the parser recovered.
ok = not any(e[1] == 2 and e[3] and e[2][0] for e in d.events[j:])
# A saved network skips provisioning: the board must start unprovisioned.
ok &= ask(d, "state", 2, lambda e: state(e) and e[2][0] != 4)
if any(state(e) and e[2][0] == 4 for e in d.events[j:]):
# The rest of the flow means nothing on a provisioned board; stop before re-provisioning it.
print(
"board is already provisioned: erase the flash first (a new config name keeps the saved network)"
)
print("FAIL")
sys.exit(1)
ok &= ask(d, "info", 3, rpc(3))
ok &= ask(d, "netstate", 7, rpc(7))
# The scan sends one reply per network and ends with an empty one.
ok &= ask(d, "scan", 4, lambda e: rpc(4)(e) and e[2][1] == 0, timeout=10)
seen = any(e[1] == 4 and e[3] and e[2][0] == 4 and s in e[2] for e in d.events)
print(f"target network in scan: {seen}")
creds = bytes([len(s)]) + s
if not o.skip_bad:
bad_pw = b"wrong-password-0123456789"
ok &= ask(
d,
"provision bad password",
1,
lambda e: e[1] == 2 and e[3] and e[2][0] == 3,
timeout=60,
payload=creds + bytes([len(bad_pw)]) + bad_pw,
)
d.pump(2)
# common.yaml sets next_url, so the provisioned reply must carry a URL.
ok &= ask(
d,
"provision good",
1,
lambda e: (
rpc(1)(e) and any(x.startswith("http") for x in strings(rpc_strings(e[2])))
),
timeout=60,
payload=creds + bytes([len(w)]) + w,
)
d.pump(3)
ok &= ask(d, "state is provisioned", 2, lambda e: state(e) and e[2][0] == 4)
ok &= ask(d, "netstate", 7, rpc(7))
d.pump(2)
bad = sum(1 for e in d.events if not e[3])
# Errors outside an ask() window, boot included: only the bad password case may report one.
errors = sum(1 for e in d.events if e[1] == 2 and e[3] and e[2][0])
if errors > (0 if o.skip_bad else 1):
print(f"unexpected error frames: {errors}")
ok = False
reboots = d.boots - boots_at_start
if (d.reopens or reboots) and not o.allow_reset:
ok = False
print(
f"frames={len(d.events)} badframes={bad} reopens={d.reopens} reboots={reboots} loglines={d.loglines} "
f"(before stray {before})"
)
print("PASS" if ok and not bad else "FAIL")
sys.exit(0 if ok and not bad else 1)
if __name__ == "__main__":
main()
@@ -1,23 +0,0 @@
#!/bin/bash
# run.sh NAME [improvcli.py args...]: build NAME.yaml (next to this script, or a path) from TREE, erase the flash so no
# credentials are saved, flash it over serial and run improvcli.py on PORT. Output lands in out-NAME/.
# env: TREE (checkout under test), PORT, PY (checkout python), SECRETS (secrets.yaml with wifi_ssid, wifi_password),
# BUILD_LOCK (optional); builds go through ../pio_build.sh with a private PlatformIO directory
# ESP chips only (esptool erase). On LibreTiny or RP2040 erase and flash with that platform's tools, then run
# improvcli.py directly. Exit status is improvcli.py's: 0 pass, 1 fail, 2 setup failed.
set -o pipefail
D=$(cd "$(dirname "$0")" && pwd); N=${1:?name}; shift
W=${TREE:?set TREE}; PORT=${PORT:?set PORT}; S=${SECRETS:?set SECRETS}; PY=${PY:-$W/venv/bin/python}
Y=$N; [ -f "$Y" ] || Y=$D/$N.yaml; Y=$(cd "$(dirname "$Y")" && pwd)/$(basename "$Y"); N=$(basename "$Y" .yaml)
O=$D/out-$N; mkdir -p "$O"
REV=$(git -C "$W" rev-parse --short=10 HEAD) || { echo "TREE is not a git checkout: $W"; exit 2; }
echo "== $N $REV"
# the erase runs while the firmware compiles
$PY -m esptool --port "$PORT" erase_flash > "$O/erase.log" 2>&1 &
ERASE=$!
"$(dirname "$D")/pio_build.sh" compile "$N" "$Y" || { wait $ERASE; exit 2; }
wait $ERASE || { echo "$N erase failed"; exit 2; }
"$(dirname "$D")/pio_build.sh" upload "$N" "$Y" "$PORT" || exit 2
sleep 1
rm -f "$O/raw.bin"
$PY "$D/improvcli.py" "$PORT" "$O/raw.bin" --secrets "$S" "$@" 2>&1 | tee "$O/improv.txt"
@@ -1,22 +0,0 @@
"""lognorm.py [FILE...]: print ESPHome log lines without ANSI colours, the capture's `[hh:mm:ss]` or
`[hh:mm:ss.mmm]` timestamp and the `:NNN` source line number of the tag, so two builds' logs can be diffed.
Reads stdin when no file is given. Scripts can import normalize() instead.
"""
import fileinput
import re
ANSI = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
STAMP = re.compile(r"^\[\d\d:\d\d:\d\d(?:\.\d+)?\]")
LINENO = re.compile(r"(\[[A-Z]\]\[[\w.\-]+):\d+\]")
def normalize(line: str) -> str:
return LINENO.sub(r"\1]", STAMP.sub("", ANSI.sub("", line)))
if __name__ == "__main__":
with fileinput.input(encoding="utf-8", errors="replace") as fh:
for raw in fh:
print(normalize(raw.rstrip("\n")))
@@ -1,20 +0,0 @@
# XIAO nRF54LM20A with every nPM1300 entity: dumps the registers and switch states 8 s after boot, then runs the
# toggles script. Name set with -s name.
esphome:
name: ${name}
on_boot:
- priority: -100
then:
- delay: 8s
- script.execute: regdump
- script.execute: swdump
- delay: 2s
- script.execute: toggles
nrf52:
board: xiao_nrf54lm20a
logger:
level: DEBUG
hardware_uart: UART0
packages:
regdump: !include regdump.yaml
common: !include npmcommon.yaml
@@ -1,179 +0,0 @@
# nPM1300 hub with every entity, a 500 mA VBUS limit, the charger settings, BUCK2 at 3.3 V and LDO1 as a 3.3 V LDO,
# state log lines from on_value and on_state lambdas, a register and switch dump every 60 s, and a toggles script
# (charger off and on, LDO1 off and on). Never switch BUCK2 off on the XIAO nRF54LM20A: it powers the board.
i2c:
- id: pmic_bus
sda: P1.18
scl: P1.17
npm1300:
id: pmic
i2c_id: pmic_bus
update_interval: 5s
vbus_current_limit: 500mA
thermistor_resistance: 10kOhm
thermistor_beta: 3380
charger:
charge_current: 150mA
termination_voltage: 4.2V
warm_termination_voltage: 4.0V
termination_current: 20%
trickle_voltage: 2.5V
discharge_current_limit: 1000mA
recharge: true
buck2:
voltage: 3.3V
ldo1:
voltage: 3.3V
mode: ldo
enabled: true
ldo2:
mode: load_switch
enabled: false
globals:
- id: txt_count
type: int
initial_value: '0'
sensor:
- platform: npm1300
battery_voltage:
name: VBAT
on_value:
- lambda: 'ESP_LOGI("st", "SENS VBAT=%.4f", x);'
battery_current:
name: IBAT
on_value:
- lambda: 'ESP_LOGI("st", "SENS IBAT=%.4f", x);'
battery_temperature:
name: TBAT
on_value:
- lambda: 'ESP_LOGI("st", "SENS TBAT=%.4f", x);'
die_temperature:
name: TDIE
on_value:
- lambda: 'ESP_LOGI("st", "SENS TDIE=%.4f", x);'
system_voltage:
name: VSYS
on_value:
- lambda: 'ESP_LOGI("st", "SENS VSYS=%.4f", x);'
vbus_voltage:
name: VBUS
on_value:
- lambda: 'ESP_LOGI("st", "SENS VBUS=%.4f", x);'
vbus_current_limit:
name: ILIM
on_value:
- lambda: 'ESP_LOGI("st", "SENS ILIM=%.4f", x);'
binary_sensor:
- platform: npm1300
vbus_present:
name: VBUS Present
trigger_on_initial_state: true
on_state:
- lambda: 'ESP_LOGI("st", "BIN vbus_present=%s", x ? "ON" : "OFF");'
vbus_current_limit_active:
name: ILIM Active
trigger_on_initial_state: true
on_state:
- lambda: 'ESP_LOGI("st", "BIN vbus_current_limit_active=%s", x ? "ON" : "OFF");'
charging:
name: Charging
trigger_on_initial_state: true
on_state:
- lambda: 'ESP_LOGI("st", "BIN charging=%s", x ? "ON" : "OFF");'
charge_complete:
name: Charge Complete
trigger_on_initial_state: true
on_state:
- lambda: 'ESP_LOGI("st", "BIN charge_complete=%s", x ? "ON" : "OFF");'
battery_detected:
name: Battery Detected
trigger_on_initial_state: true
on_state:
- lambda: 'ESP_LOGI("st", "BIN battery_detected=%s", x ? "ON" : "OFF");'
charger_error:
name: Charger Error
trigger_on_initial_state: true
on_state:
- lambda: 'ESP_LOGI("st", "BIN charger_error=%s", x ? "ON" : "OFF");'
text_sensor:
- platform: npm1300
charger_status:
name: Charger Status
on_value:
- lambda: |-
id(txt_count) += 1;
ESP_LOGI("st", "TXT #%d charger_status='%s'", id(txt_count), x.c_str());
switch:
- platform: npm1300
charger:
id: sw_charger
name: Charger
restore_mode: DISABLED
on_state:
- lambda: 'ESP_LOGI("st", "SW charger=%s", x ? "ON" : "OFF");'
buck1:
id: sw_buck1
name: BUCK1
restore_mode: DISABLED
on_state:
- lambda: 'ESP_LOGI("st", "SW buck1=%s", x ? "ON" : "OFF");'
buck2:
id: sw_buck2
name: BUCK2
restore_mode: DISABLED
on_state:
- lambda: 'ESP_LOGI("st", "SW buck2=%s", x ? "ON" : "OFF");'
ldo1:
id: sw_ldo1
name: LDO1
restore_mode: DISABLED
on_state:
- lambda: 'ESP_LOGI("st", "SW ldo1=%s", x ? "ON" : "OFF");'
ldo2:
id: sw_ldo2
name: LDO2
restore_mode: DISABLED
on_state:
- lambda: 'ESP_LOGI("st", "SW ldo2=%s", x ? "ON" : "OFF");'
script:
- id: swdump
then:
- lambda: |-
ESP_LOGI("st", "SWSTATE charger=%d buck1=%d buck2=%d ldo1=%d ldo2=%d txt_count=%d", id(sw_charger).state,
id(sw_buck1).state, id(sw_buck2).state, id(sw_ldo1).state, id(sw_ldo2).state, id(txt_count));
- id: toggles
then:
- logger.log: "STEP charger off"
- switch.turn_off: sw_charger
- delay: 1s
- script.execute: regdump
- component.update: pmic
- delay: 1s
- script.execute: swdump
- logger.log: "STEP charger on"
- switch.turn_on: sw_charger
- delay: 1s
- script.execute: regdump
- component.update: pmic
- delay: 1s
- script.execute: swdump
- logger.log: "STEP ldo1 off"
- switch.turn_off: sw_ldo1
- delay: 1s
- script.execute: regdump
- component.update: pmic
- delay: 1s
- script.execute: swdump
- logger.log: "STEP ldo1 on"
- switch.turn_on: sw_ldo1
- delay: 1s
- script.execute: regdump
- component.update: pmic
- delay: 1s
- script.execute: swdump
- logger.log: "STEP done"
interval:
- interval: 60s
then:
- script.execute: regdump
- script.execute: swdump
@@ -1,38 +0,0 @@
# XIAO nRF54LM20A probe build: the hub with no regulator or charger settings, a bus scan and a register dump every
# 20 s, to see the power-on state before writing anything. Name set with -s name.
esphome:
name: ${name}
on_boot:
- priority: -100
then:
- delay: 3s
- script.execute: regdump
nrf52:
board: xiao_nrf54lm20a
logger:
level: DEBUG
hardware_uart: UART0
packages:
regdump: !include regdump.yaml
i2c:
- id: pmic_bus
sda: P1.18
scl: P1.17
scan: true
npm1300:
id: pmic
i2c_id: pmic_bus
sensor:
- platform: npm1300
system_voltage:
name: VSYS
vbus_voltage:
name: VBUS
battery_voltage:
name: VBAT
die_temperature:
name: Die
interval:
- interval: 20s
then:
- script.execute: regdump
@@ -1,16 +0,0 @@
# Register dump of the nPM1300 with id(pmic).read_register16; "!" marks a failed read.
script:
- id: regdump
then:
- lambda: |-
static const uint16_t REGS[] = {0x0201, 0x0202, 0x0207, 0x0304, 0x0308, 0x0309, 0x030C, 0x0334, 0x0336,
0x0408, 0x040A, 0x040F, 0x0410, 0x0411, 0x0434, 0x0804, 0x0807, 0x0808,
0x0809, 0x080C, 0x080D, 0x050A};
char buf[400];
size_t pos = 0;
for (uint16_t r : REGS) {
uint8_t v = 0;
auto err = id(pmic).read_register16(r, &v, 1);
pos += snprintf(buf + pos, sizeof(buf) - pos, " %04X=%s%02X", r, err == i2c::ERROR_OK ? "" : "!", v);
}
ESP_LOGI("regs", "REGS%s", buf);
@@ -1,28 +0,0 @@
#!/bin/bash
# runone.sh NAME CONFIG PORT [SECS]: flash CONFIG as NAME with `esphome run --no-logs` (pyOCD through the XIAO's
# CMSIS-DAP probe) from TREE while a serial capture of SECS (default 240) stays open on the probe's PORT. The first
# seconds of the capture are the old firmware; cut at "Log initialized". A failed compile, flash or timeout, or no
# "compiled on" line with this build's time in the capture, exits 1. env: TREE, PY, PIO_CORE
D=$(cd "$(dirname "$0")" && pwd); S=$(dirname "$D"); N=${1:?name}; C=${2:?config}; P=${3:?port}; SECS=${4:-240}
T=${TREE:?set TREE}; PY=${PY:-$T/venv/bin/python}
export PLATFORMIO_CORE_DIR=${PIO_CORE:-$D/pio} PYTHONPATH=$T PYTHONUNBUFFERED=1
$PY "$S/serial_capture.py" "$P" "$SECS" "$D/$N-cap.txt" > /dev/null 2> "$D/$N-cap.err" &
CAP=$!
sleep 2
if ! (cd "$T" && timeout 300 $PY -m esphome -s name "$N" run --no-logs "$D/$C" --device "$P") > "$D/$N-run.log" 2>&1
then
# The capture holds the old firmware only; stop it so it is not read as a run of the new build.
tail -1 "$D/$N-run.log"; kill $CAP 2>/dev/null; wait $CAP 2>/dev/null
echo "FAIL: $N run failed (see $D/$N-run.log)"; exit 1
fi
tail -1 "$D/$N-run.log"
# A capture that ended during the flash holds only the old firmware
kill -0 $CAP 2>/dev/null || { echo "FAIL: the capture ended before the flash finished (see $D/$N-cap.err; if empty, raise SECS)"; exit 1; }
wait $CAP
# a flash that returned 0 but never booted the new image leaves only the old firmware in the capture, so look for
# this build's own compile time, which every boot logs
BI=$D/.esphome/build/$N/src/esphome/core/build_info_data.cpp
BT=$(sed -n 's/.*ESPHOME_BUILD_TIME_STR\[\][^"]*"\(.*\)";.*/\1/p' "$BI" 2>/dev/null | head -1)
[ -n "$BT" ] || { echo "FAIL: no build time in $BI"; exit 1; }
grep -a -q -F "compiled on $BT" "$D/$N-cap.txt" ||
{ echo "FAIL: no boot of the build from $BT in $D/$N-cap.txt"; exit 1; }
@@ -1,27 +0,0 @@
#!/bin/bash
# pio_build.sh compile|upload NAME YAML [DEVICE] [esphome options...]: esphome compile or upload of YAML with
# `-s name NAME`, built from TREE with a private PlatformIO core directory. Logs to <yaml dir>/NAME-<action>.log and
# prints the RAM and Flash lines (compile) or the upload result. Exits 1 on a failure or when esphome would not import
# from TREE. A config that sets `name: ${name}` builds one copy per NAME, so a PR and a dev build of the same file
# never share a build directory.
# env: TREE (checkout under test), PY (default $TREE/venv/bin/python), PIO_CORE (default <yaml dir>/pio),
# BUILD_LOCK (optional command prefix that serialises builds on a shared machine; compile only)
A=${1:?compile|upload}; N=${2:?name}; Y=${3:?yaml}; shift 3
T=${TREE:?set TREE}; PY=${PY:-$T/venv/bin/python}
Y=$(cd "$(dirname "$Y")" && pwd)/$(basename "$Y"); D=$(dirname "$Y")
export PLATFORMIO_CORE_DIR=${PIO_CORE:-$D/pio} PYTHONPATH=$T PYTHONUNBUFFERED=1
L=$D/$N-$A.log
F=$(cd "$T" && $PY -c 'import esphome, os; print(os.path.dirname(esphome.__file__))')
[ "$F" = "$(cd "$T" && pwd)/esphome" ] || { echo "esphome imports from $F, not $T"; exit 1; }
echo "esphome from $F" > "$L"
case $A in
compile)
(cd "$T" && ${BUILD_LOCK:-} timeout 1500 $PY -m esphome -s name "$N" "$@" compile "$Y") >> "$L" 2>&1
RC=$?; grep -a -E "^(RAM|Flash):" "$L"; [ $RC = 0 ] || { echo "$N compile failed (see $L)"; exit 1; } ;;
upload)
DEV=${1:?device}; shift
(cd "$T" && timeout 600 $PY -m esphome -s name "$N" "$@" upload "$Y" --device "$DEV") >> "$L" 2>&1 ||
{ echo "$N upload failed (see $L)"; tail -3 "$L"; exit 1; }
grep -a -E "OTA successful|Successfully uploaded" "$L" | tail -1 ;;
*) echo "unknown action $A"; exit 1 ;;
esac
@@ -1,16 +1,8 @@
# Shared by pa.yaml and pc.yaml: Pico 2 W, pioscope.hpp, PIO1 memory cleared at boot, an ALIVE line every 2 s and
# the s_ws_p0 strip on pin 2.
# Shared by pa.yaml and pc.yaml: Pico 2 W, pioscope.hpp, an ALIVE line every 2 s and the s_ws_p0 strip on pin 2.
esphome:
name: ${name}
includes:
- pioscope.hpp
on_boot:
# PIO instruction memory survives a reflash; clear PIO1 so a program left by the previous firmware cannot hide a
# missing pio_add_program
- priority: 1000
then:
- lambda: |-
for (int i = 0; i < 32; i++) pio1->instr_mem[i] = 0;
rp2040:
board: rpipico2w
@@ -1,33 +0,0 @@
esphome:
name: ${name}
esp32:
board: esp32-c3-devkitm-1
framework:
type: esp-idf
logger:
level: DEBUG
hardware_uart: USB_SERIAL_JTAG
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
api:
reboot_timeout: 0s
encryption:
key: !secret api_key
ota:
- platform: esphome
spi:
clk_pin: GPIO6
mosi_pin: GPIO7
miso_pin: GPIO10
cc1101:
id: radio
cs_pin: GPIO3
frequency: 433.92MHz
remote_receiver:
pin: GPIO4
dump: all
interval:
- interval: 30s
then:
- lambda: ESP_LOGI("alive", "up %u heap %u", (unsigned) (millis() / 1000), (unsigned) esp_get_free_heap_size());
@@ -1,22 +0,0 @@
#!/bin/bash
# cmp.sh BASE_DEBUG PR_DEBUG BASE_VERBOSE PR_VERBOSE: compare host harness outputs from the first INPUT line on,
# with colours, timestamps and line numbers removed (../../lognorm.py). Writes db/dp/vb/vp.txt to the current directory.
PY=${PY:-python3}; LN=$(cd "$(dirname "$0")/../.." && pwd)/lognorm.py
TMP=$(mktemp -d); trap 'rm -rf "$TMP"' EXIT
[ $# -eq 4 ] || { echo "usage: cmp.sh BASE_DEBUG PR_DEBUG BASE_VERBOSE PR_VERBOSE"; exit 1; }
i=0
for f in "$1" "$2" "$3" "$4"; do
i=$((i + 1))
[ -s "$f" ] || { echo "missing or empty: $f"; exit 1; }
"$PY" "$LN" "$f" | awk '/^INPUT /{f=1} f' > "$TMP/$i"
# no INPUT line: the harness did not run, and two such files would compare as identical
[ -s "$TMP/$i" ] || { echo "no INPUT lines in $f"; exit 1; }
done
NSYM='s/^\[[DV]\]\[remote.beo4\]: Beo4: n_sym/[*][remote.beo4]: Beo4: n_sym/'
sed -E "$NSYM" "$TMP/3" > vb.txt
sed -E "$NSYM" "$TMP/4" > vp.txt
echo "VERBOSE diff (n_sym level mapped): $(diff vb.txt vp.txt | grep -c '^[<>]') of $(wc -l < vp.txt)"
grep -v 'Beo4: n_sym' "$TMP/1" > db.txt; cp "$TMP/2" dp.txt
echo "DEBUG diff (base n_sym dropped): $(diff db.txt dp.txt | grep -c '^[<>]') of $(wc -l < dp.txt)"
echo "n_sym: DEBUG base $(grep -c '\[D\]\[remote.beo4\]: Beo4: n_sym' "$TMP/1") pr $(grep -c 'Beo4: n_sym' dp.txt);" \
"VERBOSE base D $(grep -c '\[D\]\[remote.beo4\]: Beo4: n_sym' "$TMP/3") pr V $(grep -c '\[V\]\[remote.beo4\]: Beo4: n_sym' "$TMP/4")"
@@ -1,44 +0,0 @@
"""gen_dumpers.py TREE [OUT]: write OUT (default dumpers.h here) for rbh.hpp, one dumper per DECLARE_REMOTE_PROTOCOL(...) in TREE's
remote_base plus Pronto, RCSwitch and ABBWelcome (Raw is called by rbh.hpp only when nothing matched)."""
import pathlib
import re
import sys
rb = pathlib.Path(sys.argv[1]) / "esphome" / "components" / "remote_base"
headers, names = [], []
for h in sorted(rb.glob("*_protocol.h")):
found = re.findall(
r"^DECLARE_REMOTE_PROTOCOL\((\w+)\)",
h.read_text(encoding="utf-8"),
re.MULTILINE,
)
if found or h.name in (
"pronto_protocol.h",
"rc_switch_protocol.h",
"abbwelcome_protocol.h",
"raw_protocol.h",
):
headers.append(h.name)
names += found
names += [n for n in ("Pronto", "RCSwitch", "ABBWelcome") if n not in names]
out = ["#pragma once"]
out += [f'#include "esphome/components/remote_base/{h}"' for h in headers]
out += [
"using namespace esphome::remote_base;",
"struct RbDumper {",
" const char *name;",
" RemoteReceiverDumperBase *d;",
"};",
]
out += [f"static {n}Dumper rbd_{n.lower()};" for n in names]
out += ["static RbDumper RB_DUMPERS[] = {"]
out += [f' {{"{n}", &rbd_{n.lower()}}},' for n in names]
out += ["};", ""]
out_path = (
pathlib.Path(sys.argv[2])
if len(sys.argv) > 2
else pathlib.Path(__file__).with_name("dumpers.h")
)
out_path.write_text("\n".join(out), encoding="utf-8")
print(len(names), "dumpers")
@@ -1,99 +0,0 @@
esphome:
name: ${name}
on_boot:
priority: -100
then:
- logger.set_level: DEBUG
logger:
level: VERBOSE
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
power_save_mode: none
api:
reboot_timeout: 0s
encryption:
key: !secret api_key
ota:
- platform: esphome
remote_transmitter:
id: irtx
pin:
number: ${pin}
allow_other_uses: true
carrier_duty_percent: 100%
remote_receiver:
pin:
number: ${pin}
allow_other_uses: true
dump: all
idle: 25ms
button:
- platform: template
name: TX all
on_press:
- script.execute: tx_all
- platform: template
name: Verbose
on_press:
- logger.set_level: VERBOSE
- platform: template
name: Debug
on_press:
- logger.set_level: DEBUG
script:
- id: tx_all
mode: single
then:
- logger.log: "TXMARK aok 1"
- lambda: ${prep}
- remote_transmitter.transmit_aok:
remote_id: 0x123456
channels: 0x00A1
command: 0x0B
- delay: 500ms
- logger.log: "TXMARK aok 2"
- lambda: ${prep}
- remote_transmitter.transmit_aok:
remote_id: 0xA5C381
channels: 0x1000
command: 0x43
- delay: 500ms
- logger.log: "TXMARK cardin 1"
- lambda: ${prep}
- remote_transmitter.transmit_cardin_s438:
code: "01001001110100"
channel: B
- delay: 700ms
- logger.log: "TXMARK cardin 2"
- lambda: ${prep}
- remote_transmitter.transmit_cardin_s438:
code: "11100011000101"
channel: A
- delay: 700ms
- logger.log: "TXMARK drayton 1"
- lambda: ${prep}
- remote_transmitter.transmit_drayton:
address: 0x1230
data: 0x8
- delay: 500ms
- logger.log: "TXMARK drayton 2"
- lambda: ${prep}
- remote_transmitter.transmit_drayton:
address: 0x1234
channel: 0x01
command: 0x05
- delay: 500ms
- logger.log: "TXMARK beo4 1"
- lambda: ${prep}
- remote_transmitter.transmit_beo4:
source: 0x01
command: 0x0C
- delay: 500ms
- logger.log: "TXMARK beo4 2"
- lambda: ${prep}
- remote_transmitter.transmit_beo4:
source: 0x00
command: 0x60
- delay: 500ms
- logger.log: "TXMARK end"
@@ -1,240 +0,0 @@
#pragma once
#include <cerrno>
#include <cstdio>
#include <cstdlib>
#include <random>
#include <string>
#include <vector>
#include "dumpers.h"
static std::mt19937 rbh_rng(22814);
static uint32_t rbh_r() { return rbh_rng(); }
static RawDumper rbh_raw;
static void rbh_dump_all(const RawTimings &raw, uint32_t tol, ToleranceMode mode, const char *what, int i) {
printf("INPUT %d %s tol=%u mode=%d n=%u\n", i, what, (unsigned) tol, (int) mode, (unsigned) raw.size());
bool any = false;
for (auto &nd : RB_DUMPERS) {
RemoteReceiveData src(raw, tol, mode);
bool r = nd.d->dump(src);
if (r) {
printf("HIT %s\n", nd.name);
any = true;
}
}
if (!any) {
RemoteReceiveData src(raw, tol, mode);
rbh_raw.dump(src);
}
RemoteReceiveData s1(raw, tol, mode);
auto a = AOKProtocol().decode(s1);
if (a)
printf("AOK id=%06X ch=%04X cmd=%02X\n", (unsigned) a->remote_id, a->channels, a->command);
RemoteReceiveData s2(raw, tol, mode);
auto c = CardinS438Protocol().decode(s2);
if (c)
printf("CARDIN code=%04X ch=%u\n", c->code, c->channel);
RemoteReceiveData s3(raw, tol, mode);
auto d = DraytonProtocol().decode(s3);
if (d)
printf("DRAYTON addr=%04X ch=%u cmd=%u data=%u\n", d->address, d->channel, d->command, d->data);
RemoteReceiveData s4(raw, tol, mode);
auto b = Beo4Protocol().decode(s4);
if (b)
printf("BEO4 src=%u cmd=%u rep=%u\n", b->source, b->command, b->repeats);
}
struct Mode {
uint32_t tol;
ToleranceMode mode;
};
static const Mode RBH_MODES[] = {{25, TOLERANCE_MODE_PERCENTAGE},
{10, TOLERANCE_MODE_PERCENTAGE},
{150, TOLERANCE_MODE_TIME},
{60, TOLERANCE_MODE_TIME}};
// One mark or space nudged to just inside or just outside the tolerance of its own length
static int32_t rbh_edge(int32_t v, const Mode &m) {
int32_t n = v < 0 ? -v : v;
int32_t lo, hi;
if (m.mode == TOLERANCE_MODE_TIME) {
lo = n - (int32_t) m.tol;
hi = n + (int32_t) m.tol;
} else {
lo = int32_t(100 - m.tol) * n / 100;
hi = int32_t(100 + m.tol) * n / 100;
}
int32_t pick[] = {lo - 1, lo, hi, hi + 1};
int32_t e = pick[rbh_r() % 4];
if (e < 1)
e = 1;
return v < 0 ? -e : e;
}
static RawTimings rbh_frame(int proto) {
RemoteTransmitData t;
switch (proto) {
case 0:
AOKProtocol().encode(&t, AOKData{rbh_r() & 0xFFFFFF, (uint16_t) rbh_r(), (uint8_t) rbh_r()});
break;
case 1: {
CardinS438Data c;
c.code = rbh_r();
c.channel = rbh_r() % 4;
c.seed = rbh_r();
c.repeats = 1 + rbh_r() % 3;
CardinS438Protocol().encode(&t, c);
break;
}
case 2:
DraytonProtocol().encode(&t, DraytonData{(uint16_t) rbh_r(), 0, 0, (uint8_t) (rbh_r() & 0x7F)});
break;
default:
Beo4Protocol().encode(&t, Beo4Data{(uint8_t) rbh_r(), (uint8_t) rbh_r(), 0});
break;
}
// A receiver sees back to back marks (or spaces) as one longer timing
RawTimings merged;
for (int32_t v : t.get_data()) {
if (!merged.empty() && (merged.back() > 0) == (v > 0))
merged.back() += v;
else
merged.push_back(v);
}
return merged;
}
static void rbh_noise(RawTimings &raw, int n) {
bool mark = raw.empty() || raw.back() < 0;
for (int i = 0; i < n; i++, mark = !mark) {
int32_t v = 50 + rbh_r() % 12000;
raw.push_back(mark ? v : -v);
}
}
static const char *RBH_PROTO[] = {"aok", "cardin", "drayton", "beo4"};
struct RbhInput {
std::string what;
int mode;
RawTimings raw;
};
static void rbh_generate(std::vector<RbhInput> &out) {
for (int rep = 0; rep < 250; rep++) {
for (int proto = 0; proto < 4; proto++) {
RawTimings f = rbh_frame(proto);
for (int variant = 0; variant < 9; variant++) {
RawTimings raw;
if (variant == 1 || variant == 4)
rbh_noise(raw, 1 + rbh_r() % 30); // leading noise
if (!raw.empty() && raw.back() > 0 && f[0] > 0)
raw.push_back(-(int32_t) (3000 + rbh_r() % 9000));
int copies = (variant == 2 || variant == 4) ? 2 + rbh_r() % 2 : 1; // repeats
for (int c = 0; c < copies; c++) {
if (c > 0 && raw.back() > 0)
raw.push_back(-(int32_t) (5000 + rbh_r() % 8000));
raw.insert(raw.end(), f.begin(), f.end());
}
for (int mi = 0; mi < 4; mi++) {
const auto &m = RBH_MODES[mi];
RawTimings x = raw;
if (variant == 3) { // nudge about a quarter of the timings to the tolerance edges
for (auto &v : x)
if (rbh_r() % 4 == 0)
v = rbh_edge(v, m);
} else if (variant >= 5 && proto < 3) { // only the sync mark, at one tolerance edge, the rest exact
const int32_t nominal[] = {5000, 6400, 1000};
int32_t n = nominal[proto], lo, hi;
if (m.mode == TOLERANCE_MODE_TIME) {
lo = n - (int32_t) m.tol;
hi = n + (int32_t) m.tol;
} else {
lo = int32_t(100 - m.tol) * n / 100;
hi = int32_t(100 + m.tol) * n / 100;
}
const int32_t edge[] = {lo - 1, lo, hi, hi + 1};
for (auto &v : x) {
if (v == n) {
v = edge[variant - 5];
if (proto == 2)
break; // Drayton: the first 1000 us mark is the sync
}
}
} else if (variant == 4) { // jitter every timing by up to 30%
for (auto &v : x) {
int32_t n = v < 0 ? -v : v;
n = n + (int32_t) (n * ((int) (rbh_r() % 61) - 30) / 100);
if (n < 1)
n = 1;
v = v < 0 ? -n : n;
}
}
x.push_back(-20000);
char what[32];
snprintf(what, sizeof(what), "%s/v%d", RBH_PROTO[proto], variant);
out.push_back({what, mi, x});
}
}
}
}
for (int r = 0; r < 4000; r++) { // random timing corpus
RawTimings raw;
rbh_noise(raw, 4 + rbh_r() % 250);
raw.push_back(-20000);
out.push_back({"random", r % 4, raw});
}
}
static void rbh_fail(const char *what, const char *path) {
fprintf(stderr, "RBH_CORPUS %s: %s\n", what, path);
exit(1);
}
// RBH_CORPUS=<file>: replay the file when it exists, else generate a corpus and write it there. Any read, parse or
// write error exits 1, so two builds never compare different inputs.
static void rbh_run() {
std::vector<RbhInput> in;
const char *path = getenv("RBH_CORPUS");
errno = 0;
FILE *f = path ? fopen(path, "rb") : nullptr;
if (path != nullptr && f == nullptr && errno != ENOENT)
rbh_fail("cannot open", path);
if (f != nullptr) {
char what[64];
int mode;
unsigned n;
int got;
while ((got = fscanf(f, "%63s %d %u", what, &mode, &n)) == 3) {
if (mode < 0 || mode >= (int) (sizeof(RBH_MODES) / sizeof(RBH_MODES[0])))
rbh_fail("bad mode in", path);
RawTimings raw(n);
for (auto &v : raw) {
if (fscanf(f, "%d", &v) != 1)
rbh_fail("truncated frame in", path);
}
in.push_back({what, mode, raw});
}
if (got != EOF || ferror(f) || in.empty())
rbh_fail("unreadable or empty", path);
fclose(f);
} else {
rbh_generate(in);
if (path != nullptr) {
if ((f = fopen(path, "wb")) == nullptr)
rbh_fail("cannot create", path);
for (auto &x : in) {
fprintf(f, "%s %d %u", x.what.c_str(), x.mode, (unsigned) x.raw.size());
for (auto v : x.raw)
fprintf(f, " %d", v);
fprintf(f, "\n");
}
if (ferror(f) || fclose(f) != 0)
rbh_fail("write failed", path);
}
}
int i = 0;
for (auto &x : in)
rbh_dump_all(x.raw, RBH_MODES[x.mode].tol, RBH_MODES[x.mode].mode, x.what.c_str(), i++);
fflush(stdout);
exit(0);
}
@@ -1,20 +0,0 @@
# build twice: `-s level DEBUG` and `-s level VERBOSE` (with a different `-s name`), then compare with cmp.sh
substitutions:
level: DEBUG
esphome:
name: ${name}
includes:
- dumpers.h
- rbh.hpp
on_boot:
priority: -100
then:
- lambda: rbh_run();
host:
logger:
level: ${level}
external_components:
- source:
type: local
path: ../ext
rbtest:
@@ -1,13 +0,0 @@
"""Test only: pulls remote_base with every protocol source into a host build (hostharness and dumpall)."""
from esphome.components import remote_base
import esphome.config_validation as cv
AUTO_LOAD = ["remote_base"]
DOMAIN = "rbtest"
CONFIG_SCHEMA = cv.Schema({})
async def to_code(config):
for stem in remote_base._PROTOCOL_STEMS:
remote_base.request_protocol(stem)
@@ -1,108 +0,0 @@
esphome:
name: ${name}
includes:
- rbh.hpp
logger:
level: DEBUG
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
power_save_mode: none
api:
reboot_timeout: 0s
encryption:
key: !secret api_key
ota:
- platform: esphome
remote_transmitter:
id: irtx
pin:
number: ${pin}
allow_other_uses: true
carrier_duty_percent: 100%
remote_receiver:
pin:
number: ${pin}
allow_other_uses: true
dump: [aeha, haier, mirage]
on_aeha:
then:
- lambda: rbh_on_frame("ON_AEHA", x.data);
on_haier:
then:
- lambda: rbh_on_frame("ON_HAIER", x.data);
on_mirage:
then:
- lambda: rbh_on_frame("ON_MIRAGE", x.data);
button:
- platform: template
name: TX all
on_press:
- script.execute: tx_all
script:
- id: tx_all
mode: single
then:
- logger.log: "TXMARK aeha 2"
- lambda: ${prep}
- remote_transmitter.transmit_aeha:
address: 0xE6C2
data: [0x79, 0xB7]
- delay: 900ms
- logger.log: "TXMARK aeha 3"
- lambda: ${prep}
- remote_transmitter.transmit_aeha:
address: 0x7EDA
data: [0x61, 0x89, 0x59]
- delay: 900ms
- logger.log: "TXMARK aeha 8"
- lambda: ${prep}
- remote_transmitter.transmit_aeha:
address: 0x9D2C
data: [0xEC, 0xD4, 0x26, 0x32, 0xB9, 0xA1, 0x14, 0xD7]
- delay: 900ms
- logger.log: "TXMARK aeha 13"
- lambda: ${prep}
- remote_transmitter.transmit_aeha:
address: 0x3E1C
data: [0xAE, 0x41, 0x17, 0xB0, 0xDC, 0x2C, 0xF8, 0x86, 0xF4, 0xF3, 0xDC, 0xD0, 0x92]
- delay: 900ms
- logger.log: "TXMARK aeha 20"
- lambda: ${prep}
- remote_transmitter.transmit_aeha:
address: 0xDD3D
data: [0x30, 0x65, 0xC9, 0x64, 0xB7, 0x32, 0xD3, 0xF5, 0x21, 0x6A, 0x62, 0x88, 0xE2, 0x5A, 0x59, 0x35, 0xF4, 0xC1, 0xC2, 0xE4]
- delay: 900ms
- logger.log: "TXMARK aeha 34"
- lambda: ${prep}
- remote_transmitter.transmit_aeha:
address: 0x9C21
data: [0x27, 0x6B, 0x15, 0x4D, 0x81, 0x14, 0x08, 0xCD, 0x36, 0xC1, 0x82, 0x3F, 0x85, 0xFE, 0x9D, 0xDD, 0x5D, 0xF9, 0x11, 0x98, 0x22, 0x88, 0x42, 0xEF, 0x73, 0xFD, 0xD7, 0x3A, 0x96, 0x50, 0xA1, 0x4E, 0xDB, 0x8D]
- delay: 900ms
- logger.log: "TXMARK aeha 35"
- lambda: ${prep}
- remote_transmitter.transmit_aeha:
address: 0x1050
data: [0x8F, 0xCA, 0xE5, 0x0A, 0x97, 0xF9, 0x34, 0xD7, 0x06, 0x64, 0x96, 0x52, 0xBE, 0x49, 0xFB, 0xB7, 0xA8, 0x46, 0xBA, 0x27, 0x8B, 0x9E, 0xA1, 0x8F, 0x6F, 0xAF, 0x97, 0xA2, 0xD1, 0x67, 0x88, 0x72, 0x8F, 0xC1, 0xD0]
- delay: 900ms
- logger.log: "TXMARK haier 13"
- lambda: ${prep}
- remote_transmitter.transmit_haier:
code: [0x22, 0x48, 0x40, 0x36, 0x31, 0x7A, 0x21, 0x08, 0x57, 0x0C, 0x7B, 0x16, 0x3A]
- delay: 600ms
- logger.log: "TXMARK haier 8"
- lambda: ${prep}
- remote_transmitter.transmit_haier:
code: [0x0C, 0xA0, 0x1C, 0x67, 0x3D, 0x0D, 0xF7, 0x81]
- delay: 600ms
- logger.log: "TXMARK mirage 14"
- lambda: ${prep}
- remote_transmitter.transmit_mirage:
code: [0x41, 0x1E, 0xF7, 0xA9, 0x30, 0xCC, 0x0B, 0xA1, 0x47, 0x86, 0xE8, 0xFF, 0x8C, 0xCF]
- delay: 600ms
- logger.log: "TXMARK mirage 14"
- lambda: ${prep}
- remote_transmitter.transmit_mirage:
code: [0x68, 0x72, 0x38, 0xAB, 0xA8, 0xF9, 0x4F, 0x86, 0x74, 0xA1, 0x39, 0x0A, 0x49, 0xC8]
- delay: 600ms
- logger.log: "TXMARK end"
@@ -1,97 +0,0 @@
#pragma once
#include <cstdio>
#include <cstdlib>
#include <random>
#include <type_traits>
#include <vector>
#include "esphome/components/remote_base/aeha_protocol.h"
#include "esphome/components/remote_base/haier_protocol.h"
#include "esphome/components/remote_base/mirage_protocol.h"
#include "esphome/core/alloc_helpers.h"
#include "esphome/core/log.h"
using namespace esphome::remote_base;
inline std::mt19937 rbh_rng(22773);
inline uint32_t rbh_r() { return rbh_rng(); }
template<typename D> void rbh_print(const char *proto, int i, const char *kind, const esphome::optional<D> &d) {
printf("CASE %s %d %s has=%d", proto, i, kind, d.has_value() ? 1 : 0);
if (d.has_value()) {
printf(" n=%u data=", (unsigned) d->data.size());
for (uint8_t b : d->data)
printf("%02X", b);
}
printf("\n");
}
// kind 0 clean, 1 one timing corrupted, 2 truncated
inline RawTimings rbh_mutate(RawTimings raw, int kind) {
raw.push_back(-20000); // the idle gap a receiver records after the last mark
if (kind == 1 && !raw.empty()) {
size_t k = rbh_r() % raw.size();
int32_t v = 100 + rbh_r() % 6000;
raw[k] = raw[k] < 0 ? -v : v;
} else if (kind == 2 && raw.size() > 1) {
raw.resize(rbh_r() % raw.size());
}
return raw;
}
inline const char *const RBH_KIND[] = {"clean", "corrupt", "trunc"};
// Encode one frame with enc, mutate it by kind, decode it, print the case and dump what decoded.
template<typename P, typename Enc>
void rbh_case(P &proto, const char *name, const char *prefix, int i, int kind, Enc enc) {
RemoteTransmitData t;
enc(&t);
RawTimings raw = rbh_mutate(t.get_data(), kind);
auto d = proto.decode(RemoteReceiveData(raw, 25, TOLERANCE_MODE_PERCENTAGE));
printf("%s", prefix);
rbh_print(name, i, RBH_KIND[kind], d);
if (!d.has_value())
return;
if constexpr (std::is_same_v<P, AEHAProtocol>)
printf("ADDR %04X\n", d->address);
proto.dump(*d);
}
inline void rbh_run() {
AEHAProtocol aeha;
HaierProtocol haier;
MirageProtocol mirage;
char prefix[48];
int i = 0;
for (int rep = 0; rep < 400; rep++) {
for (int kind = 0; kind < 3; kind++) {
uint8_t buf[48];
// AEHA: 1 to 40 bytes (35 is the cap)
size_t len = 1 + rbh_r() % 40;
for (auto &b : buf)
b = rbh_r();
uint16_t addr = rbh_r();
snprintf(prefix, sizeof(prefix), "AEHA len=%u addr=%04X ", (unsigned) len, addr);
rbh_case(aeha, "aeha", prefix, i, kind, [&](RemoteTransmitData *t) { aeha.encode(t, addr, buf, len); });
// Haier: 8 and 13 mostly, sometimes 1 to 16
size_t hl = (rep % 4 == 3) ? 1 + rbh_r() % 16 : ((rbh_r() & 1) ? 8 : 13);
snprintf(prefix, sizeof(prefix), "HAIER len=%u ", (unsigned) hl);
rbh_case(haier, "haier", prefix, i, kind, [&](RemoteTransmitData *t) { haier.encode(t, buf, hl); });
// Mirage: 14 mostly, sometimes 10 to 18
size_t ml = (rep % 4 == 3) ? 10 + rbh_r() % 9 : 14;
snprintf(prefix, sizeof(prefix), "MIRAGE len=%u ", (unsigned) ml);
rbh_case(mirage, "mirage", prefix, i, kind, [&](RemoteTransmitData *t) { mirage.encode(t, buf, ml); });
i++;
}
}
fflush(stdout);
exit(0);
}
// Loopback trigger log (ir-common.yaml): also checks that the vector idioms compile against the trigger's data type.
template<typename T> void rbh_on_frame(const char *tag, const T &data) {
std::vector<uint8_t> v = data;
uint32_t s = 0;
for (auto b : data)
s += b;
bool eq = (v == data) && (data == v) && !(data != v);
ESP_LOGI("loop", "%s n=%u hex=%s sum=%u eq=%d front=%u", tag, (unsigned) data.size(),
esphome::format_hex(data).c_str(), (unsigned) s, eq ? 1 : 0, (unsigned) data.front());
}
@@ -1,16 +0,0 @@
esphome:
name: ${name}
includes:
- rbh.hpp
on_boot:
priority: -100
then:
- lambda: rbh_run();
host:
logger:
level: DEBUG
external_components:
- source:
type: local
path: ../ext
rbtest:
@@ -1,29 +0,0 @@
"""press.py HOST BUTTON[:SECS]...: press each button (by object id) in order, waiting SECS (default 1) after each.
hostharness: `press.py HOST tx_all:14 tx_all:14 ...`; dumpall: `press.py HOST tx_all:9 tx_all:9 verbose tx_all:9 debug`;
pronto/run.sh: every tx button ROUNDS times, then `soak:90`.
API_KEY in the env selects Noise (see apiconn.py).
"""
import asyncio
import pathlib
import sys
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from aioesphomeapi import ButtonInfo # noqa: E402
from apiconn import connect # noqa: E402
async def main() -> None:
c = await connect(sys.argv[1], tries=60)
ents, _ = await c.list_entities_services()
keys = {e.object_id: e.key for e in ents if isinstance(e, ButtonInfo)}
for step in sys.argv[2:]:
name, _, secs = step.partition(":")
c.button_command(keys[name])
await asyncio.sleep(float(secs or 1))
await c.disconnect()
print("pressed", len(sys.argv) - 2)
asyncio.run(main())
@@ -1,36 +0,0 @@
"""consist.py EXT: per frame, the trigger text (decode) must equal the dump chunks joined."""
import sys
from extract import read_lines, trigger_blocks
DUMP_START = "remote.pronto: Received Pronto: data="
DUMP = "remote.pronto: "
lines = read_lines(sys.argv[1])
ok = bad = 0
for n, t, i in trigger_blocks(lines):
# dump: all prints NEC lines and the pronto binary sensor can log between the trigger and the Pronto dump
while i < len(lines) and (
lines[i].startswith("remote.nec") or "BS pronto" in lines[i]
):
i += 1
if i < len(lines) and lines[i] == DUMP_START:
i += 1
d = ""
while (
i < len(lines) and lines[i].startswith(DUMP) and "Received" not in lines[i]
):
d += lines[i][len(DUMP) :]
i += 1
if d == t and len(t) == n:
ok += 1
else:
bad += 1
print("MISMATCH", n, len(t), len(d))
else:
bad += 1
print("MISSING dump after a trigger of", n)
print(f"{ok + bad} trigger frames, {ok} consistent, {bad} not")
if bad or not ok:
sys.exit(1)
@@ -1,16 +0,0 @@
# D1 mini, one-pin loopback on GPIO12 with every dumper; the board and network come from climate_ir_loop.
packages:
base: !include ../../climate_ir_loop/base.yaml
board: !include ../../climate_ir_loop/esp8266-ir.yaml
common: !include gen-common.yaml
logger:
baud_rate: 460800
level: DEBUG
remote_receiver:
dump: all
buffer_size: 3kb
on_pronto:
- lambda: |-
ESP_LOGI("trig", "TRIG len=%u", (unsigned) x.data.size());
for (size_t i = 0; i < x.data.size(); i += 200)
ESP_LOGI("trig", "TRIG %s", x.data.substr(i, 200).c_str());
@@ -1,19 +0,0 @@
# ESP32 devkit, one-pin loopback on GPIO27 with every dumper; the board and network come from climate_ir_loop.
packages:
base: !include ../../climate_ir_loop/base.yaml
board: !include
file: ../../climate_ir_loop/esp32-ir.yaml
vars:
rmt_symbols: "256"
receive_symbols: "256"
common: !include gen-common.yaml
logger:
baud_rate: 921600
level: DEBUG
remote_receiver:
dump: all
on_pronto:
- lambda: |-
ESP_LOGI("trig", "TRIG len=%u", (unsigned) x.data.size());
for (size_t i = 0; i < x.data.size(); i += 200)
ESP_LOGI("trig", "TRIG %s", x.data.substr(i, 200).c_str());
@@ -1,64 +0,0 @@
"""extract.py SERIAL OUT: keep MARK, pronto dump, trigger, binary sensor and heap lines from the first MARK on,
with timestamps and source line numbers removed (lognorm.py). Also holds the trigger block parser that consist.py
and jit.py share."""
import pathlib
import re
import sys
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
from lognorm import normalize # noqa: E402
TRIG_LEN = "trig: TRIG len="
TRIG = "trig: TRIG "
def trigger_blocks(lines):
"""Yield (n, text, i) per on_pronto trigger: the length it logged, its chunks joined, and the index of the first
line after the block."""
i = 0
while i < len(lines):
if not lines[i].startswith(TRIG_LEN):
i += 1
continue
n = int(lines[i][len(TRIG_LEN) :])
i += 1
text = ""
while (
i < len(lines)
and lines[i].startswith(TRIG)
and not lines[i].startswith(TRIG_LEN)
):
text += lines[i][len(TRIG) :]
i += 1
yield n, text, i
def read_lines(path):
return pathlib.Path(path).read_text(encoding="utf-8").splitlines()
def main() -> None:
out, on = [], False
with pathlib.Path(sys.argv[1]).open(encoding="utf-8", errors="replace") as fh:
for raw in fh:
m = re.match(r"\[(\w)\]\[([\w.]+)\]: ?(.*)$", normalize(raw.rstrip("\n")))
if not m:
continue
_lvl, tag, msg = m.groups()
if "MARK tx" in msg:
on = True
if not on:
continue
if (
tag in ("remote.pronto", "trig", "heap")
or "MARK" in msg
or "BS pronto" in msg
or tag == "remote.nec"
):
out.append(f"{tag}: {msg}")
pathlib.Path(sys.argv[2]).write_text("\n".join(out) + "\n", encoding="utf-8")
if __name__ == "__main__":
main()
@@ -1,86 +0,0 @@
"""gen.py [OUT]: write OUT (default gen-common.yaml next to this script) with raw frames, buttons, on_pronto trigger,
pronto binary sensors and heap logging; prints each frame's length and the pronto binary sensor data."""
import pathlib
import sys
def frame(n, seed):
out = []
for i in range(n):
if i % 2 == 0:
out.append(300 + (i * 37 + seed * 101) % 900)
else:
out.append(-(400 + (i * 53 + seed * 59) % 1500))
return out
def pronto(t):
nums = [0, 4145146 // 38000, (len(t) + 1) // 2, 0]
for x in t:
d = x - 20 if x > 0 else -x + 20
nums.append((d + 13) // 26)
return " ".join(f"{v:04X}" for v in nums) + " 0181 "
frames = {
"raw21a": frame(21, 1),
"raw21b": [x + 260 if x > 0 else x - 260 for x in frame(21, 1)],
"raw41": frame(41, 2),
"raw43": frame(43, 3),
"raw121": frame(121, 4),
"raw245": frame(245, 5),
}
y = []
y.append("button:")
y.append(
" - {platform: template, name: tx nec, on_press: [{logger.log: 'MARK tx nec'}, {remote_transmitter.transmit_nec: {address: 0x1234, command: 0x78}}]}"
)
for k, v in frames.items():
y.append(
f" - platform: template\n name: tx {k}\n on_press:\n - logger.log: 'MARK tx {k}'\n - remote_transmitter.transmit_raw:\n code: {v}"
)
y.append(
""" - platform: template
name: soak
on_press:
- lambda: id(heap_log).execute("before");
- repeat:
count: 100
then:
- remote_transmitter.transmit_nec: {{address: 0x1234, command: 0x78}}
- delay: 150ms
- remote_transmitter.transmit_raw:
code: {}
- delay: 250ms
- delay: 1s
- lambda: id(heap_log).execute("after");
""".format(frames["raw245"])
)
y.append(f"""script:
- id: heap_log
parameters: {{tag: string}}
then:
- lambda: |-
#ifdef USE_ESP32
ESP_LOGI("heap", "HEAP %s free=%u largest=%u", tag.c_str(), (unsigned) heap_caps_get_free_size(MALLOC_CAP_INTERNAL), (unsigned) heap_caps_get_largest_free_block(MALLOC_CAP_INTERNAL));
#else
ESP_LOGI("heap", "HEAP %s free=%u largest=%u", tag.c_str(), (unsigned) ESP.getFreeHeap(), (unsigned) ESP.getMaxFreeBlockSize());
#endif
binary_sensor:
- platform: remote_receiver
name: pronto a
receiver_id: irrx
pronto:
data: "{pronto(frames["raw21a"]).strip()}"
on_press: [{{logger.log: 'BS pronto a ON'}}]
""")
out = (
pathlib.Path(sys.argv[1])
if len(sys.argv) > 1
else pathlib.Path(__file__).with_name("gen-common.yaml")
)
out.write_text("\n".join(y), encoding="utf-8")
for k, v in frames.items():
print(k, len(v), (4 + len(v)) * 5)
print("PRONTO_A", pronto(frames["raw21a"]))
@@ -1,31 +0,0 @@
"""jit.py EXT_A EXT_B: compare the on_pronto trigger frames of two extract.py outputs in order; counts identical
frames, frames whose length differs, and the largest difference in Pronto units (timing jitter) between same-length
frames, leaving out the header and the trailing idle."""
import sys
from extract import read_lines, trigger_blocks
def frames(p):
return [
[int(x, 16) for x in text.split()]
for _n, text, _i in trigger_blocks(read_lines(p))
]
a, b = frames(sys.argv[1]), frames(sys.argv[2])
same = sum(x == y for x, y in zip(a, b, strict=False))
lens = sum(len(x) != len(y) for x, y in zip(a, b, strict=False))
md = max(
(
abs(p - q)
for x, y in zip(a, b, strict=False)
if len(x) == len(y)
for p, q in zip(x[4:-1], y[4:-1], strict=False)
),
default=0,
)
print(
f"{len(a)}/{len(b)} frames, {same} identical, {lens} length differs, max unit diff {md}"
)
@@ -1,25 +0,0 @@
#!/bin/bash
# run.sh NAME TREE YAML PORT BAUD ROUNDS: flash NAME, capture serial, press each tx button ROUNDS times with
# ../press.py, then the soak button, and count the markers, triggers and binary sensor presses on serial
S=$(cd "$(dirname "$0")" && pwd); K=$(dirname "$(dirname "$S")"); N=$1; T=$2; Y=$3; P=$4; B=$5; R=$6
PY=${PY:-$T/venv/bin/python}
pkill -f "[s]erial_capture.py $P "
TREE=$T PIO_CORE=${PIO_CORE:-$S/pio} bash "$K/pio_build.sh" upload "$N" "$Y" "$P" || exit 1
rm -f "$S/$N-serial.txt"
# boot and IP lookup (up to 160 s), 7 presses of 1.5 s per round, the 90 s soak, margin; stopped after the drive
"$PY" "$K/serial_capture.py" "$P" $((160 + R * 11 + 120)) "$S/$N-serial.txt" --baud "$B" --reset > /dev/null 2>&1 &
CAP=$!
IP=""; for _ in $(seq 60); do IP=$(grep -a -o 'IP Address: [0-9.]*' "$S/$N-serial.txt" 2>/dev/null | tail -1 | cut -d' ' -f3); [ -n "$IP" ] && break; sleep 2; done
[ -n "$IP" ] || IP=$(bash "$K/resolve.sh" "$N" 20) || { echo no ip; kill $CAP; exit 1; }
sleep 5
PRESSES=()
for _ in $(seq "$R"); do
for b in tx_nec tx_raw21a tx_raw21b tx_raw41 tx_raw43 tx_raw121 tx_raw245; do PRESSES+=("$b:1.5"); done
done
"$PY" "$S/../press.py" "$IP" "${PRESSES[@]}" soak:90 > "$S/$N-drive.txt" 2>&1; RC=$?
sleep 3; kill $CAP; wait $CAP 2>/dev/null
cat "$S/$N-drive.txt"
echo "MARK tx $(grep -a -c 'MARK tx' "$S/$N-serial.txt") triggers $(grep -a -c 'TRIG len=' "$S/$N-serial.txt")" \
"binary sensor on_press $(grep -a -c 'BS pronto a ON' "$S/$N-serial.txt")"
[ $RC -eq 0 ] || echo "press.py failed (rc=$RC)"
exit $RC
@@ -1,10 +0,0 @@
#!/bin/bash
# resolve.sh NAME [TRIES]: print the address of NAME.local once it resolves and answers a ping, retrying every 3 s
# (default 30 tries). Exits 1 when it never does.
for _ in $(seq 1 "${2:-30}"); do
IP=$(python3 -c 'import socket, sys; print(socket.gethostbyname(sys.argv[1]))' "$1.local" 2>/dev/null) &&
ping -c 1 "$IP" > /dev/null 2>&1 && { echo "$IP"; exit 0; }
sleep 3
done
echo "$1.local did not resolve or answer" >&2
exit 1
@@ -1,19 +0,0 @@
# RP2 scheduler lock under IRQ and lwIP load (#22688)
On RP2, lwIP callbacks and pico-sdk timers can call into the scheduler from IRQ context while the main loop is inside
it. `components/irq_sched_stress/` reproduces that on purpose: a 300 us pico-sdk repeating timer calls `defer()` and
`set_timeout()` while the main loop does the same. Judge it by counts: every defer runs, and exactly 10 of 400 named
timeouts on 10 shared names run. RP2 is `ESPHOME_THREAD_SINGLE`, so `defer()` order is not FIFO on any build; the
out of order count is informational. Keep `esphome: debug_scheduler:` off, it logs from the IRQ.
- `stress.yaml`: the component, `run_test`, `run_defer` and `run_cancel` API actions, REST entities for the churn
loops and a 16 ms interval that logs its gaps. `race.yaml` ports the removed item race integration fixture.
`picow.yaml` and `evb.yaml` add them to the `../rp2_web_ble/` role configs (encrypted API, web_server, diagnostics);
put `secrets.yaml` here or next to the role configs.
- `flow.sh NAME CONFIG HOST [SERIAL_PORT]`: upload, then the actions, connstress, REST churn and both together; ends
with PASS or FAIL. `svcrun.py HOST ACTION DONE_REGEX TIMEOUT` runs one action and prints its result lines.
- `restchurn.sh HOST SECS OUT`: 4 parallel REST POST loops. `looptime.sh NAME HOST SECS`: 16 ms interval gaps and
50 ms re-arms with the API and page loads running, to compare the cost of the lock between builds.
Build both heads with `../pio_build.sh` under different names. Dev before #22688 failed the defer count on the Pico W;
the PR passed every action.
@@ -1,20 +0,0 @@
"""Test only: a pico-sdk repeating timer calls defer() and set_timeout() from IRQ context while the main loop
does the same, like the host scheduler fixtures do from std::thread workers (RP2040/RP2350 only)."""
import esphome.codegen as cg
import esphome.config_validation as cv
from esphome.const import CONF_ID
from esphome.types import ConfigType
DOMAIN = "irq_sched_stress"
irq_sched_stress_ns = cg.esphome_ns.namespace("irq_sched_stress")
IrqSchedStress = irq_sched_stress_ns.class_("IrqSchedStress", cg.Component)
CONFIG_SCHEMA = cv.Schema({cv.GenerateID(): cv.declare_id(IrqSchedStress)}).extend(
cv.COMPONENT_SCHEMA
)
async def to_code(config: ConfigType) -> None:
var = cg.new_Pvariable(config[CONF_ID])
await cg.register_component(var, config)
@@ -1,124 +0,0 @@
#pragma once
// Test only. run_defer(): N defer() calls from a 300 us repeating timer IRQ plus N from the main loop; every one must
// run. run_cancel(): NC set_timeout() calls on 10 shared names from each side inside one 150 ms window; exactly 10
// must run, the rest are implicit cancellations. Results are logged as "DEFER RESULT" and "CANCEL RESULT".
// Keep esphome: debug_scheduler off, it logs from the IRQ.
#include "esphome/core/component.h"
#include "esphome/core/hal.h"
#include "esphome/core/log.h"
#include <pico/time.h>
namespace esphome::irq_sched_stress {
class IrqSchedStress : public Component {
public:
void loop() override {
if (this->mode_ == IDLE || this->finishing_)
return;
// Issue the main loop share in one tight burst while the IRQ issues its share, so every operation lands inside
// the 150 ms timeout window and the IRQ interleaves with the main loop inside the scheduler.
// The time limit keeps a stalled timer IRQ from wedging the main loop; the result then reports the shortfall.
uint32_t lim = this->mode_ == DEFER ? N : NC;
while ((this->main_issued_ < lim || this->irq_issued_ < lim) && millis() - this->start_ms_ < ISSUE_LIMIT_MS) {
if (this->main_issued_ < lim)
this->step_(false);
delayMicroseconds(60);
}
cancel_repeating_timer(&this->timer_);
this->issue_ms_ = millis() - this->start_ms_;
this->finishing_ = true;
this->set_timeout(1000, [this]() { this->finish_(); });
}
void run_defer() { this->start_(DEFER); }
void run_cancel() { this->start_(CANCEL); }
protected:
enum Mode : uint8_t { IDLE, DEFER, CANCEL };
static constexpr uint32_t N = 500; // per source, defer mode
static constexpr uint32_t NC = 200; // per source, cancel mode
static constexpr uint32_t ISSUE_LIMIT_MS = 2000;
static constexpr const char *NAMES[10] = {"shared_0", "shared_1", "shared_2", "shared_3", "shared_4",
"shared_5", "shared_6", "shared_7", "shared_8", "shared_9"};
static bool irq_tick_(repeating_timer_t *t) {
static_cast<IrqSchedStress *>(t->user_data)->step_(true);
return true;
}
void start_(Mode mode) {
if (this->mode_ != IDLE) {
// a request during a running test is dropped; say so instead of leaving the caller to time out
ESP_LOGW("irq_sched_stress", "%s RESULT FAIL busy", mode == DEFER ? "DEFER" : "CANCEL");
return;
}
this->irq_issued_ = this->main_issued_ = this->irq_exec_ = this->main_exec_ = 0;
this->fifo_viol_ = this->cancel_exec_ = 0;
this->irq_last_ = this->main_last_ = -1;
this->finishing_ = false;
this->start_ms_ = millis();
ESP_LOGI("irq_sched_stress", "%s test start", mode == DEFER ? "defer" : "cancel");
this->mode_ = mode;
if (!add_repeating_timer_us(-300, &IrqSchedStress::irq_tick_, this, &this->timer_)) {
this->mode_ = IDLE;
ESP_LOGE("irq_sched_stress", "%s RESULT FAIL timer not started", mode == DEFER ? "DEFER" : "CANCEL");
}
}
// One operation from the IRQ (irq true) or the main loop.
void step_(bool irq) {
volatile uint32_t &issued = irq ? this->irq_issued_ : this->main_issued_;
if (this->mode_ == DEFER && issued < N) {
int32_t id = issued;
issued = issued + 1;
if (irq) {
this->defer([this, id]() { this->ran_(this->irq_last_, this->irq_exec_, id); });
} else {
this->defer([this, id]() { this->ran_(this->main_last_, this->main_exec_, id); });
}
} else if (this->mode_ == CANCEL && issued < NC) {
uint32_t i = issued;
issued = issued + 1;
this->set_timeout(NAMES[i % 10], 150, [this]() { this->cancel_exec_ = this->cancel_exec_ + 1; });
}
}
void ran_(volatile int32_t &last, volatile uint32_t &exec, int32_t id) {
if (id != last + 1)
this->fifo_viol_ = this->fifo_viol_ + 1;
last = id;
exec = exec + 1;
}
void finish_() {
unsigned total = this->irq_issued_ + this->main_issued_;
if (this->mode_ == DEFER) {
bool ok = this->irq_exec_ == N && this->main_exec_ == N;
ESP_LOGI("irq_sched_stress", "DEFER RESULT %s irq_exec=%u/%u main_exec=%u/%u out_of_order=%u issued_in=%u ms",
ok ? "PASS" : "FAIL", (unsigned) this->irq_exec_, (unsigned) N, (unsigned) this->main_exec_,
(unsigned) N, (unsigned) this->fifo_viol_, (unsigned) this->issue_ms_);
} else {
// Without every set_timeout() issued from both sides the race never ran; 10 executions alone prove nothing.
bool ok = this->cancel_exec_ == 10 && this->irq_issued_ == NC && this->main_issued_ == NC;
ESP_LOGI("irq_sched_stress", "CANCEL RESULT %s scheduled=%u executed=%u (expect 10) issued_in=%u ms",
ok ? "PASS" : "FAIL", total, (unsigned) this->cancel_exec_, (unsigned) this->issue_ms_);
}
this->mode_ = IDLE;
}
repeating_timer_t timer_{};
volatile Mode mode_{IDLE};
volatile uint32_t irq_issued_{0};
volatile uint32_t main_issued_{0};
volatile uint32_t irq_exec_{0};
volatile uint32_t main_exec_{0};
volatile int32_t irq_last_{-1};
volatile int32_t main_last_{-1};
volatile uint32_t fifo_viol_{0};
volatile uint32_t cancel_exec_{0};
uint32_t start_ms_{0};
uint32_t issue_ms_{0};
bool finishing_{false};
};
} // namespace esphome::irq_sched_stress
@@ -1,5 +0,0 @@
# W5100S-EVB-Pico on wired Ethernet with the scheduler lock packages (name set with -s name).
packages:
board: !include ../rp2_web_ble/evb.yaml
stress: !include stress.yaml
race: !include race.yaml
@@ -1,65 +0,0 @@
#!/bin/zsh
# flow.sh NAME CONFIG HOST [SERIAL_PORT]: the RP2 scheduler lock flow (#22688). Uploads CONFIG (picow.yaml or evb.yaml)
# as NAME to DEVICE (default HOST, an OTA), captures SERIAL_PORT for the whole run when given, polls the Fast counter,
# then runs the removed item race, IRQ defer and IRQ cancel actions, connstress 500x4 and 500x2, 90 s of REST churn,
# REST churn and connstress together, and the IRQ actions again. PASS needs every action to pass, connstress 500x2
# without a failure, at least half of each REST churn answered 200, at least 90% of the counter polls answered, no
# reboot (counter reset) and, with SERIAL_PORT, no crash or assert line and a capture that lasted the run; exits 1
# otherwise. connstress at 4 is diagnostic: the API has 3 sockets on these builds, so 4 at a time always shows some
# "dropped after hello" failures.
# env: TREE, PY, DEVICE, API_KEY (unset for a plaintext API)
D=${0:A:h}; N=$1; C=$2; H=$3; SP=$4; S=$D/..; cd $D
T=${TREE:?set TREE}; PY=${PY:-$T/venv/bin/python}; L=$D/$N-flow.txt; : > $L
FAILS=0; fail() { echo "FAIL: $*" | tee -a $L; FAILS=$((FAILS + 1)); }
log() { echo "$*" | tee -a $L; }
# restchurn.sh prints "REST 200:n 404:m ..."; fail when fewer than half of the POSTs answered 200.
restok() {
local ok all
read ok all < <(echo "$2" | tr ' ' '\n' | awk -F: 'NF == 2 {a += $2; if ($1 == 200) o = $2} END {print o+0, a+0}')
[ "$all" -gt 0 ] && [ $((ok * 2)) -ge "$all" ] || fail "$1: $ok of $all POSTs answered 200"; }
heap() { curl -s -m 4 "http://$H/sensor/Heap%20free" | grep -o '"value":[0-9]*' | cut -d: -f2; }
# Portable TCP check of the API port (nc -G is macOS only): apiopen TIMEOUT_S
apiopen() { $PY -c 'import socket, sys; socket.create_connection((sys.argv[1], 6053), float(sys.argv[2])).close()' $H $1 2>/dev/null; }
up() { for _ in $(seq 1 60); do apiopen 1 && return 0; sleep 2; done; return 1; }
$S/pio_build.sh upload $N $D/$C ${DEVICE:-$H} || exit 1
up || { echo "FAIL: API port never opened"; exit 1; }; sleep 8
if [ -n "$SP" ]; then $PY $S/serial_capture.py $SP 3600 $D/$N-serial.log 2> $D/$N-cap.err & CAP=$!; sleep 3; fi
( while true; do echo "$(date +%s) $(curl -s -m 3 "http://$H/sensor/Fast%20counter" | grep -o '"value":[0-9]*')"; sleep 2; done ) > $D/$N-ctr.txt 2>&1 &
CTR=$!; trap 'kill $CTR $CAP 2>/dev/null' EXIT
actions() {
for a in "run_test:Test Complete:20" "run_defer:DEFER RESULT:30" "run_cancel:CANCEL RESULT:30"; do
IFS=: read s pat t <<< "$a"; [ "$1" = again ] && [ $s = run_test ] && continue
out=$($PY -I $D/svcrun.py $H $s "$pat" $t 2>&1) || fail "$s ($1)"; log "$out"; sleep 2
done
}
log "heap at start: $(heap)"
actions first
$PY -I $S/api/connstress.py $H 500 4 > $D/$N-cs4.txt 2>&1; log "connstress 500x4: $(grep -a -E '^Success|^RESULT' $D/$N-cs4.txt | tr '\n' ' ')"
sleep 5
$PY -I $S/api/connstress.py $H 500 2 > $D/$N-cs2.txt 2>&1 || fail "connstress 500x2"
log "connstress 500x2: $(grep -a -E '^Success|^RESULT' $D/$N-cs2.txt | tr '\n' ' ')"
sleep 5; R=$($D/restchurn.sh $H 90 $D/$N-rest.txt); log "REST churn 90 s: $R"; restok "REST churn 90 s" "$R"
sleep 5; log "heap after REST churn: $(heap)"
$D/restchurn.sh $H 120 $D/$N-rest2.txt > $D/$N-rest2.sum & RP=$!
$PY -I $S/api/connstress.py $H 500 4 > $D/$N-csrest.txt 2>&1
log "connstress 500x4 during REST churn: $(grep -a -E '^Success|^RESULT' $D/$N-csrest.txt | tr '\n' ' ')"
wait $RP; R=$(cat $D/$N-rest2.sum); log "REST during connstress: $R"; restok "REST during connstress" "$R"
sleep 5; log "heap at end: $(heap)"
actions again
[ -n "$SP" ] && { kill -0 $CAP 2>/dev/null || fail "the capture ended before the run did (see $D/$N-cap.err)"; }
kill $CTR $CAP 2>/dev/null; wait $CAP 2>/dev/null; trap - EXIT
RESETS=$(grep -o 'value":[0-9]*' $D/$N-ctr.txt | cut -d: -f2 | awk 'NR>1 && $1<p {r++} {p=$1} END {print r+0}')
POLLS=$(wc -l < $D/$N-ctr.txt | tr -d ' '); BADPOLLS=$(grep -vc value $D/$N-ctr.txt)
log "counter resets (reboots): $RESETS, failed counter polls: $BADPOLLS of $POLLS"
[ "$RESETS" = 0 ] || fail "$RESETS reboots"
# A wedged web_server or a board that is down answers no polls, which would otherwise read as no reboot.
[ "$POLLS" -gt 0 ] && [ $((BADPOLLS * 10)) -le "$POLLS" ] || fail "$BADPOLLS of $POLLS counter polls failed"
if [ -n "$SP" ]; then
CRASH=$(grep -a -c -i -E 'CRASH|panic|HardFault|assert' $D/$N-serial.log)
log "serial: crash lines $CRASH, boots $(grep -a -c 'Log initialized' $D/$N-serial.log)"
# a live capture that read nothing would pass the crash check without proving anything
[ "$(grep -a -c -v '^\[[0-9:.]*\]\[capture\]' $D/$N-serial.log)" -gt 0 ] || fail "no firmware lines in $D/$N-serial.log"
[ "$CRASH" = 0 ] || fail "$CRASH crash or assert lines"
fi
[ $FAILS = 0 ] || exit 1
log PASS
@@ -1,22 +0,0 @@
#!/bin/zsh
# looptime.sh NAME HOST SECS: the API connected (logs and states) plus a page load every 2 s for SECS, then a summary of
# the 16 ms interval gap lines from stress.yaml (average, worst) and the 50 ms re-arms per 5 s. Diagnostic only;
# compare two builds. Exits 1 when apilog.py fails, fewer than 2 windows were logged or most page loads did not
# return 200, so a broken or unloaded run is not read as a comparison.
# env: TREE or PY, API_KEY (unset for a plaintext API)
D=${0:A:h}; N=$1; H=$2; S=$3; PY=${PY:-${TREE:?set TREE or PY}/venv/bin/python}; cd $D
# apilog.py appends, so start from an empty log or an earlier run's windows count for this one
: > $D/$N-lt-api.log
$PY -I ../rp2_web_ble/apilog.py $H $S $D/$N-lt-api.log > /dev/null & AP=$!
( end=$(( $(date +%s) + S ))
while (( $(date +%s) < end )); do curl -s -o /dev/null -m 5 -w '%{http_code}\n' http://$H/; sleep 2; done
) > $N-lt-web.txt & PP=$!
wait $AP; RC=$?; kill $PP 2>/dev/null
[ $RC = 0 ] || { echo "$N: apilog.py failed ($RC), see $N-lt-api.log"; exit 1; }
WOK=$(grep -c '^200$' $N-lt-web.txt); WALL=$(wc -l < $N-lt-web.txt | tr -d ' ')
[ "$WALL" -gt 0 ] && [ $((WOK * 2)) -ge "$WALL" ] ||
{ echo "$N: NOT LOADED, $WOK of $WALL page loads returned 200"; exit 1; }
grep -a -o "n=[0-9]* avg_gap_us=[0-9]* max_gap_us=[0-9]* btn=[0-9]* rearms=[0-9]*" $N-lt-api.log | tail -n +2 > $N-lt.txt
awk -v n=$N '{split($2,b,"="); split($3,c,"="); split($5,d,"="); k++; sa+=b[2]; if (c[2]>mx) mx=c[2]; sm+=c[2]; if (k>1) rr+=d[2]-pr; pr=d[2]}
END {if (k < 2) {print n ": fewer than 2 windows"; exit 1}
printf "%s: %d windows of 5 s, 16 ms interval avg gap %.0f us, mean window max %.0f us, worst %d us, re-arms per 5 s %.1f\n", n, k, sa/k, sm/k, mx, rr/(k-1)}' $N-lt.txt
@@ -1,10 +0,0 @@
# Pico W on the station network with the scheduler lock packages (name set with -s name).
rp2:
board: rpipicow
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
packages:
common: !include ../rp2_web_ble/rp-common.yaml
stress: !include stress.yaml
race: !include race.yaml
@@ -1,40 +0,0 @@
# The removed item race from tests/integration/fixtures/scheduler_removed_item_race.yaml, run on the board through
# the run_test action: cancelled timeouts behind a live one at the front of the heap must never run.
globals:
- id: removed_item_executed
type: int
initial_value: '0'
- id: normal_item_executed
type: int
initial_value: '0'
sensor:
- platform: template
id: race_sensor
update_interval: never
lambda: return 0.0;
script:
- id: run_test_script
then:
- lambda: |-
id(removed_item_executed) = 0;
id(normal_item_executed) = 0;
auto *c = id(race_sensor);
auto normal = []() { id(normal_item_executed)++; };
auto removed = []() { id(removed_item_executed)++; };
App.scheduler.set_timeout(c, "blocker", 1, normal);
App.scheduler.set_timeout(c, "cancel_1", 1, removed);
App.scheduler.set_timeout(c, "cancel_2", 1, removed);
App.scheduler.set_timeout(c, "cancel_3", 1, removed);
App.scheduler.set_timeout(c, "normal_1", 1, normal);
App.scheduler.set_timeout(c, "normal_2", 1, normal);
App.scheduler.set_timeout(c, "normal_3", 1, normal);
App.scheduler.process_to_add();
App.scheduler.cancel_timeout(c, "cancel_1");
App.scheduler.cancel_timeout(c, "cancel_2");
App.scheduler.cancel_timeout(c, "cancel_3");
- delay: 20ms
- lambda: |-
bool ok = id(removed_item_executed) == 0 && id(normal_item_executed) == 4;
ESP_LOGI("test", "%s: normal %d (expect 4), removed %d (expect 0)", ok ? "TEST PASSED" : "TEST FAILED",
id(normal_item_executed), id(removed_item_executed));
ESP_LOGI("test", "=== Test Complete ===");
@@ -1,10 +0,0 @@
#!/bin/zsh
# restchurn.sh HOST SECS OUT: 4 parallel loops of REST POSTs (Sw toggle, Lt on with a brightness, Num set, Btn press
# from stress.yaml) for SECS; writes "time code" lines to OUT and prints the count per HTTP code.
H=$1; S=$2; O=$3; : > $O; : > $O.1; : > $O.2; : > $O.3; : > $O.4; end=$(( $(date +%s) + S ))
# $1 is expanded per request outside the command substitution, so each $RANDOM in it is a new value
loop() { while (( $(date +%s) < end )); do u=${(e)1}; echo "$(date +%s) $(curl -s -o /dev/null -m 3 -w '%{http_code}' -X POST "http://$H$u")"; done >> $O.$2; }
loop '/switch/Sw/toggle' 1 & loop '/light/Lt/turn_on?brightness=$((RANDOM % 255))' 2 &
loop '/number/Num/set?value=$((RANDOM % 100))' 3 & loop '/button/Btn/press' 4 & wait
cat $O.1 $O.2 $O.3 $O.4 | sort -n > $O; rm -f $O.1 $O.2 $O.3 $O.4
echo "REST $(awk '{c[$2]++} END {for (k in c) printf "%s:%d ", k, c[k]}' $O)"
Loaded 100 of 730 files, more files were not shown because too many files have changed in this diff. Show more