Compare commits

..
Author SHA1 Message Date
J. Nick Koston 7a36a832e3 Simplify the while check and name its condition in one helper
The condition's name and place in the file come from one helper, also
for a list condition, without the private config location formatter.
The fix text is shared by the error and the warning, and the check
stops at the first action that waits.
2026-11-09 02:10:37 -08:00
J. Nick Koston 098891bef6 [core] Reject a while whose body never waits and cannot change its condition
A while whose body never waits never lets other components run, so a
condition only they can change stays true until the device crashes. When
the condition only reads entity state and the body neither runs code nor
acts on what the condition reads, validation now fails and says to add a
delay. Other bodies that never wait and may not change the condition get
a warning with the loop's file and line.
2026-11-09 02:07:34 -08:00
1544 changed files with 9604 additions and 57412 deletions

No files matched your search

-10
View File
@@ -40,14 +40,6 @@ Not shared code: the component's own platform files (`esphome/components/<c>/sen
`text_sensor/__init__.py`, ...), its new `USE_*` lines in `esphome/core/defines.h`,
and its `CODEOWNERS` entry.
## YAML language freeze
Until configurations move to yaml.rocks (esphome/backlog#201), flag any PR that adds to the YAML language: a new tag
in `esphome/yaml_util.py`, new behaviour in the `packages` or `substitutions` components (a new entry in the Jinja
`SAFE_GLOBALS` in `substitutions/jinja.py`, or a custom Jinja filter, counts), or new merge, override or remove
semantics (`esphome/config_helpers.py`, `!extend`, `!remove`). Bug fixes to the existing features are fine. This
covers how YAML files are read, included and combined, not a component's own configuration options.
## Memory and embedded constraints (highest value)
Heap allocation after `setup()` is treated as a reliability bug, not a
@@ -112,8 +104,6 @@ helper; `AGENTS.md` section 4 and the `component-codegen` skill have the detail.
| --- | --- |
| `if (this->x_sensor_ != nullptr) this->x_sensor_->publish_state(v);` | `PUBLISH_STATE(this->x_sensor_, v);` (`esphome/core/entity_base.h`) |
| set warning / `return` / clear warning around a bus result | `if (!this->status_comm_result(ok)) return;` |
| set warning / `return` around an earlier step of a multi-step poll | `if (!this->status_comm_check(ok)) return;` |
| warning set, but no `status_comm_result` or `status_clear_warning` on the success path | the last step uses `status_comm_result(ok)`, or the warning never clears |
| hand-written `set_x_sensor()` plus `x_sensor_` member | `SUB_SENSOR(x)`, `SUB_BINARY_SENSOR`, `SUB_TEXT_SENSOR`, `SUB_SWITCH`, ... |
| `if CONF_X in config: s = await sensor.new_sensor(config[CONF_X])` plus setter | `sensors = sensor.sub_sensors(config)` then `await sensors(CONF_X, hub.set_x_sensor)` |
| hand-written `{"a": MyEnum.MY_ENUM_A, ...}` dict | `MyEnum.options("a", "b", prefix="MY_ENUM_")` |
@@ -236,8 +236,6 @@ Never a template per bit or a virtual listener base (a second vptr per entity).
instead, so pass `buf` once and mark the component failed in `setup()` if it is nullptr. `align=4` emits
`alignas(4)`, so a word-aligned byte buffer stays `uint8`. `size` may be an expression such as `MyClass.static_("buffer_size")(n)`, which renders
`ns::MyClass::buffer_size(n)`.
- `cg.static_storage(config[CONF_ID], T, count)` is raw aligned storage for `count` objects as a `std::span<T>`,
for `StaticFixedVector::init()` when codegen knows the count.
- `display.add_writer(var, config)` sets the lambda.
- `MyEnum.options("on", prefix="MY_ENUM_")` builds an enum dict.
- `button.restart_button_schema(cls)` (and `factory_reset_`, `query_`) are button presets.
-3
View File
@@ -88,9 +88,6 @@ Components that define buses directly are flagged as "NEEDS MIGRATION" and canno
CI build time. `!extend` and `!remove` in a `test.*.yaml` also make the component ungroupable; add a
standalone entity with its own id instead.
In a grouped test, set `i2c_id: i2c_bus`, `one_wire_id` and `motion_id` even with one candidate: a merged
batch can add a second one ("Too many candidates"). `script/ci-custom.py` checks this.
A test that needs a shared component in a particular state (`esp32_ble` security settings, an option that
PSRAM rejects) cannot be grouped: the merged block wins and the whole batch fails. Add the component to
`ISOLATED_COMPONENTS` in `script/analyze_component_buses.py`, or move the option to a `test-<variant>` file.
-2
View File
@@ -107,8 +107,6 @@ APIs.
This eliminates `_M_realloc_insert` and `_M_default_append` template instantiations (200-500 bytes per
instance), needs no upper bound, and is compatible with protobuf code generation when using
`[(fixed_vector) = true]`.
When codegen knows the count, use `StaticFixedVector<T>` and pass `cg.static_storage(config[CONF_ID], T, count)`
to its `init()`, so the elements live in BSS instead of the heap.
4. **Small datasets (1-16 elements):** use `std::vector` or `std::array` with simple structs instead of
`std::map`/`std::set`/`std::unordered_map`.
+6 -44
View File
@@ -47,15 +47,6 @@ driven, what the logs show, and the same flow on a `dev` build when a quirk coul
piped, or lines land minutes late. Bound every long command with `timeout` (coreutils) or
`perl -e 'alarm shift; exec @ARGV' <seconds> <command>` on macOS without it. Save an exit code with `rc=$?`
right after the command: in `echo "$(date +%T) rc=$?"` the command substitution runs first and `$?` reads 0.
8. **Give builds a private PlatformIO directory when sessions may use different Pythons.** ESPHome stamps the
PlatformIO core directory with the Python version that last built, and a build from another version wipes and
reinstalls the shared packages under every other build. Set `PLATFORMIO_CORE_DIR=<scratch>/pio` on every
compile and upload (`scripts/pio_build.sh` does). Old releases and framework A/B builds need one too
(see `reference/boards.md`).
9. **One ESP-IDF or LibreTiny build at a time on a shared machine.** Concurrent builds race over tool caches. Wrap
compiles in a lock (`BUILD_LOCK` for `scripts/pio_build.sh`, for example a script that holds a `mkdir` lock).
10. **Check that the stimulus ran.** A load or soak run where most requests failed was not loaded, and a capture
that stopped early covers nothing; check the driver's own counters before reading the device's verdict.
## Workflow
@@ -65,9 +56,6 @@ driven, what the logs show, and the same flow on a `dev` build when a quirk coul
them apart with a 10 s capture using `serial_capture.py --reset` (ESP8266 prints its ROM banner at
74880 baud, LibreTiny boards print `[LT]` and ESPHome lines, BK7238 prints its bootloader version).
`ports.py --wait 2e8a` blocks until a Pico re-enumerates.
A change in validation or platform independent code needs no board: a `host:` build serves the API on
port 6053 on every interface (reach it at 127.0.0.1, and add `api: encryption:` on a shared network); see
recipes/homeassistant.md for one against a throwaway Home Assistant.
2. **Make a scratch directory** outside the checkout, with `secrets.yaml` and the templates you need.
One yaml per board per variant, file name equal to `name:`.
3. **Compile** in the right tree:
@@ -90,17 +78,7 @@ driven, what the logs show, and the same flow on a `dev` build when a quirk coul
> Automated testing results
```
Then one short sentence saying what was tested (builds and SHAs, board, what was driven), and on the next line a
bold result, one of **Test result: PASS**, **Test result: FAIL**, **Test result: PASS WITH CAVEATS** or
**Test result: UNCLEAR**, followed by one sentence with the key numbers (for example "the device closed none of 32
streams, dev closed 12 of 16"). FAIL means a regression or a fix that does not work. PASS WITH CAVEATS means it works
but something about the change itself needs a look; name the caveat in that sentence. A limit of the test (the
original crash did not reproduce) or a device or network quirk the tooling reported correctly is not a caveat on
the change; say it as a separate note. UNCLEAR means the run could not decide; say what is
missing. A reader must not have to dig through the table or guess. Side findings that do not change the result (a
bug already on `dev`, a one-off oddity) go in the bullets, marked as separate from the result.
Then a results table
Then one short sentence saying what was tested (builds and SHAs, board, what was driven), a results table
(build SHA, board and framework, what was driven, counts, result, plus the `dev` control row), short bullets
for the findings, and the setup and exact flow in a collapsed `<details>` block. Answer any question asked on
the PR directly in the comment. No credentials, SSIDs, IP addresses, port names or other bench details.
@@ -137,13 +115,8 @@ ESP-IDF OTA image: `.esphome/build/<name>/build/firmware.ota.bin`. LibreTiny OTA
and then; retry up to three times.
- `api_button.py HOST KEY list|press NAME [...]`: list a device's template buttons or press them over the API; the
recipes drive their test configs with it.
- `apiconn.py`: `connect(host, tries, port, on_stop)` (API connect and login with retries, `API_KEY` from the env),
`log_text()` (an API log line without ANSI codes), `stop_event()` (an Event set when the connection ends),
`ble_connect()` (wait for an advert, connect through a proxy without a cache like Home Assistant) and
`enable_notify()` (start notify and write the CCCD), imported by the API drivers.
- `pio_build.sh compile|upload NAME YAML [DEVICE] [esphome options]`: build or upload YAML as `-s name NAME` from
`TREE` with a private PlatformIO directory, an import check and an optional `BUILD_LOCK`; logs to `NAME-*.log`.
- `resolve.sh NAME [TRIES]`: the address of `NAME.local` once it resolves and answers a ping.
- `apiconn.py`: `connect(host, tries)` (API connect and login with retries, `API_KEY` from the env) and `log_text()`
(an API log line without ANSI codes), imported by the newer API drivers.
- `wifisave_hammer.sh KIND DUT_PORT DUT_YAML DUT_TREE CLIENT_PORT CLIENT_YAML CLIENT_TREE [SETTLE]`:
the fallback AP plus captive portal hammer end to end (compile, flash, capture, counts) with the
templates below. KIND is `esp`, `pico`, `bw15` or `bk`. It ends with `RESULT PASS` (exit 0),
@@ -152,13 +125,12 @@ ESP-IDF OTA image: `.esphome/build/<name>/build/firmware.ota.bin`. LibreTiny OTA
(exit 2: a precondition was missed, the stimulus was insufficient, or a capture did not last the run)
or exits 1 when a build, flash or port step failed.
Per-topic scripts, each described in the matching `reference/recipes/<component>.md`: `heap_alloc/` (operator new caller
recorder, dump saver and decoder, cover cycle driver), `online_image/`
Per-topic scripts, each described in the matching `reference/recipes/<component>.md`: `online_image/`
(server with prefixed misbehaviour cases, PNG, JPEG, BMP and QOI generators, API drivers, first pass timer), `zwave_proxy/` (checksum verifier, device info), `usb_cdc_acm/`
(host driver, Zephyr driver probe), `rtsp_server/` (protocol client, raw rate reader, measurement set,
API image grab), `cse7766/` (API observer with ping), `zephyr_coredump/` (reader flow, decode with the
Zephyr tools), `libretiny_prefs/` (rtc preference harness header), `libretiny_crash/` (BK72xx crash probe header,
capture decoder, RTL8720C ROM dump versus report matcher), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `ld2450/` (zone number driver, loopback harness driver with number and binary sensor states and TX frame summarizer, frame stream harness with presence hold and link loss drivers, BT MAC driver), `api/` (raw plaintext native API client for user defined actions, store_yaml recovery client over Noise or plaintext, `overflow/` API overflow buffer counters with a sensor config generator), `http_request/` (update manifest server, update state driver, request test server, action case driver, concurrency, reboot, update check and OTA drivers, `ota_auth/` OTA request cases with basic auth), `modem/` (button and sensor driver over the plaintext API, also used for the BLE MIDI boards, AT commands for the UART passthrough), `dfrobot_sen0395/` (API driver and case runner), `homeassistant_light/` (direct bulb driver, held-report case driver), `usb_audio/` (URL player), `speaker_media_player/` (API player driver, pipeline task case runner), `web_server/` (SSE client, request mix, SSE and HTTPD case runners, per-session `/events` listing check with its config, `/events` head, multi-client, stall, Wi-Fi drop and churn modes with their config, head comparer and runner, `async_bump/` ESPAsyncWebServer bump flows and ESP8266 web OTA cases), `mqtt/` (`ha_discovery/`, `disable/`, `retained_burst/` and `reconnect/` runners; broker config with TLS, client-certificate and WebSocket listeners, test CA with server and client certificates, transport case runner, flash-and-drive runner, CLI TLS check without a board, MAC suffix runner with per-variant configs for two boards, `shutdown/` shutdown message runner and matcher), `udp/` (datagram recorder, `udp.write` payload checker and case driver), `entity_icons/` (icon and device class readback over the API, MQTT discovery, web_server and dump_config, with a comparer), `uart_sim/` (API driver and flash, capture and drive runner for the simulated UART sensors), `remote_base/` (loopback runner and API log decoder per sent frame), `xiaomi_ble/` (frame and header generator with the expected counts, injecting, advertising and receiving configs, counter and runner), `ble_terminate/` (proxy and peer configs, connect and disconnect driver, peer disconnect matcher, runner), `i2c_slave/` (D1 mini slave and BK7238 master configs), `wpa3_ap/` (PMF AP with join and leave logging, BK7238 and CB3S DUTs, runner and summary), `wisp_ap/` (WISP upstream with AP buttons, WISP and ALWAYS_ON DUTs, client, runner and per phase summary), `wifi_restart/` (WPA3 mixed WISP upstream, static IP DUTs with syslog, runner and summary), `ota_timing/` (OTA upload timer with deflate, raw and TCP modes, API build check, A/B runner and median summary, `inflate_bench/` on-device inflate benchmark with stream generator, configs and syslog runner), `gpio_hold/` (ISR pin hold self test with ESP32 and S3 configs), `ble_event_inject/` (GAP and GATTC event injector header and proxy package), `light_sleep_wake/` (LEDC wave and edge timer for light sleep wakeup on a C3, external wave generator and per-call edge detail probe for peripheral power-down on the P4), `manual_ip/` (station and AP manual IP configs, DNS probe and runners), `web_handlers/` (late handler, web_server, captive_portal and prometheus variants, web OTA runner), `bk72xx_flash/` (flash readback with retries, per partition diff of two dumps), `ble_server_security/` (secure GATT server and pairing client configs, driver, summary and runner), `prometheus/` (datetime metrics configs and driver), `zephyr_ble_server/` (nRF52 server, looping ESP32 client and UF2 flash and count runner), `ble_adv_triggers/` (advertiser with six raw frames, ESP32 and BK72xx receivers with eleven triggers, runner and checker), `sun/` (longitude sweep package, runner and checker), `sensor_filters/` (every filter shape with a fixed input sequence, per-board packages and runner), `led_driver_sniff/` (SPI slave capture header with selftest, config generator, window comparer and runner), `hub_input_sim/` (keypad, wiegand and rdm6300 simulation on a D1 mini, API log runner), `value_range/` (on_value_range sequence with a restart, runner), `ezo_fake_bus/` (fake EZO I2C bus and step config, runner), `twai_self/` (TWAI self-reception header, on_frame config and counter runner), `ota_automations/` (OTA automation package for both OTA platforms, four case runner, repeated web upload runner and per case summary), `output_button/` (pad edge timer header, button package and runner), `homeassistant_button/` (raw plaintext API client for action calls and wizard inputs, package and runner), `uart_button/` (UART0 loopback package, API driver with TX and RX counts, runner), `interval/` (plain and delayed interval package, timing summary and runner), `smp_udp_ota/` (SMP reset fault injector, OTA runner, per mode result check, MCUboot swap limit, running image check and slot blanking UF2, ThreaLine truncated
capture decoder, RTL8720C ROM dump versus report matcher), `api_outgoing/` (wake cycle and Wi-Fi drop timing readers), `ld2450/` (zone number driver, loopback harness driver with number and binary sensor states and TX frame summarizer, frame stream harness with presence hold and link loss drivers, BT MAC driver), `api/` (raw plaintext native API client for user defined actions, store_yaml recovery client over Noise or plaintext), `http_request/` (update manifest server, update state driver, request test server, action case driver, concurrency, reboot, update check and OTA drivers, `ota_auth/` OTA request cases with basic auth), `modem/` (button and sensor driver over the plaintext API, also used for the BLE MIDI boards), `dfrobot_sen0395/` (API driver and case runner), `homeassistant_light/` (direct bulb driver, held-report case driver), `usb_audio/` (URL player), `speaker_media_player/` (API player driver, pipeline task case runner), `web_server/` (SSE client, request mix, SSE and HTTPD case runners), `mqtt/` (`ha_discovery/`, `disable/`, `retained_burst/` and `reconnect/` runners; broker config with TLS, client-certificate and WebSocket listeners, test CA with server and client certificates, transport case runner, flash-and-drive runner, CLI TLS check without a board, MAC suffix runner with per-variant configs for two boards, `shutdown/` shutdown message runner and matcher), `udp/` (datagram recorder, `udp.write` payload checker and case driver), `entity_icons/` (icon and device class readback over the API, MQTT discovery, web_server and dump_config, with a comparer), `uart_sim/` (API driver and flash, capture and drive runner for the simulated UART sensors), `remote_base/` (loopback runner and API log decoder per sent frame), `xiaomi_ble/` (frame and header generator with the expected counts, injecting, advertising and receiving configs, counter and runner), `ble_terminate/` (proxy and peer configs, connect and disconnect driver, peer disconnect matcher, runner), `i2c_slave/` (D1 mini slave and BK7238 master configs), `wpa3_ap/` (PMF AP with join and leave logging, BK7238 and CB3S DUTs, runner and summary), `wisp_ap/` (WISP upstream with AP buttons, WISP and ALWAYS_ON DUTs, client, runner and per phase summary), `wifi_restart/` (WPA3 mixed WISP upstream, static IP DUTs with syslog, runner and summary), `ota_timing/` (OTA upload timer with deflate, raw and TCP modes, API build check, A/B runner and median summary, `inflate_bench/` on-device inflate benchmark with stream generator, configs and syslog runner), `gpio_hold/` (ISR pin hold self test with ESP32 and S3 configs), `ble_event_inject/` (GAP and GATTC event injector header and proxy package), `light_sleep_wake/` (LEDC wave and edge timer for light sleep wakeup on a C3, external wave generator and per-call edge detail probe for peripheral power-down on the P4), `manual_ip/` (station and AP manual IP configs, DNS probe and runners), `web_handlers/` (late handler, web_server, captive_portal and prometheus variants, web OTA runner), `bk72xx_flash/` (flash readback with retries, per partition diff of two dumps), `ble_server_security/` (secure GATT server and pairing client configs, driver, summary and runner), `prometheus/` (datetime metrics configs and driver), `zephyr_ble_server/` (nRF52 server, looping ESP32 client and UF2 flash and count runner), `ble_adv_triggers/` (advertiser with six raw frames, ESP32 and BK72xx receivers with eleven triggers, runner and checker), `sun/` (longitude sweep package, runner and checker), `sensor_filters/` (every filter shape with a fixed input sequence, per-board packages and runner), `led_driver_sniff/` (SPI slave capture header with selftest, config generator, window comparer and runner), `hub_input_sim/` (keypad, wiegand and rdm6300 simulation on a D1 mini, API log runner), `value_range/` (on_value_range sequence with a restart, runner), `ezo_fake_bus/` (fake EZO I2C bus and step config, runner), `twai_self/` (TWAI self-reception header, on_frame config and counter runner), `ota_automations/` (OTA automation package for both OTA platforms, four case runner, repeated web upload runner and per case summary), `output_button/` (pad edge timer header, button package and runner), `homeassistant_button/` (raw plaintext API client for action calls and wizard inputs, package and runner), `uart_button/` (UART0 loopback package, API driver with TX and RX counts, runner), `interval/` (plain and delayed interval package, timing summary and runner), `smp_udp_ota/` (SMP reset fault injector, OTA runner, per mode result check, MCUboot swap limit, running image check and slot blanking UF2, Thread device config), `ld2420_sim/` (fake LD2420 answering the hub's commands on a UART0 loopback, API state recorder, two-build comparer and runner), `nrf52_tcp_probe/` (TCP and UDP socket probe config, build and log check, API check over Thread), `rs485_frame/` (simulated DLE/STX/ETX bus device, frame and gate cases, runner), `hx711/` (fake HXLine truncated
## Templates in `$SKILL/templates`
@@ -167,8 +139,6 @@ capture decoder, RTL8720C ROM dump versus report matcher), `api_outgoing/` (wake
- `client-wifisave.yaml` (package): joins that AP, posts N bogus `/wifisave` requests, then the real
credentials from `!secret`. `example-client.yaml` is the per-run file. `bogus_saves: "1"` gives a
plain provisioning check.
- `diagnostics.yaml` (package): DEBUG logs, esphome OTA, uptime, free heap, reset reason and a 200 ms counter
for soak and load tests.
- Per-topic device configs, each described in `reference/recipes/<component>.md`: `online-image-dut.yaml`,
`usb-cdc-acm-dongle-dut.yaml`, `nrf52-preferences-dut.yaml`, `rtsp-waveshare-s3-eth-dut.yaml`,
`waveshare-s3-eth-camdiag.yaml`, `s31-cse7766-dut.yaml`, `nrf52-coredump-crash-dut.yaml`,
@@ -185,8 +155,7 @@ capture decoder, RTL8720C ROM dump versus report matcher), `api_outgoing/` (wake
action package with per-board files and an OTA flash variant), `led_wire/` (DUT config and header that reads the
LED strip pads back with RMT RX and an SPI slave), `uart_sim/` (simulator header, common package, per-build parts and run files) and `remote_base/` (common
package, transmit script, per-board templates and the ESP8266 software sender), `i2c_fake/` (fake I2C device
headers with their test configs, see `fake_i2c.md`), `network_dns/` (DNS loss diagnostics for
`network: priority`, see `network_dns.md`).
headers with their test configs, see `fake_i2c.md`).
- `microphone_http_stream/`: base config, fake counter and USB microphone variants and the multi-reader client,
described in `reference/recipes/microphone_http_stream.md`.
@@ -200,9 +169,6 @@ comment quotes the exact commits. Fetch `origin dev` before computing a merge ba
## When something looks wrong
- A harness header fails to compile on Arduino or LibreTiny: those frameworks define `millis()` and `micros()` as
macros, so call `(esphome::millis)()`, which builds everywhere. Realtek SDK headers also define `OFF`, `ON`
and `SUCCESS`.
- A capture stopped early or a line is missing: check the timestamps before trusting it.
- A script that pipes into `tee` never ends although its work is done: a background command written as
`(cd dir && server & echo $!)` backgrounds the whole `cd && server` list, so a shell stays behind holding the
@@ -223,10 +189,6 @@ comment quotes the exact commits. Fetch `origin dev` before computing a merge ba
the shared `~/.platformio/platforms/libretiny`, so every later build on the machine uses the patched copy. Patch
through the ESPHome side (a linker `-Wl,-wrap,<symbol>` or a define) instead, and if it happened, restore the
directory from a clean copy of the same release.
- Copying a .uf2 to a mounted bootloader drive (`RPI-RP2`, `ITSY840BOOT`) from a terminal on macOS can fail with
"Operation not permitted" (the Removable Volumes privacy permission), and picotool then sees no device. Flash through
the serial path instead (`esphome upload --device <port>`: the 1200 baud touch on RP2, serial DFU on Adafruit nRF52
boards) or by OTA; a board that only shows the drive needs the file dragged in Finder.
- `wc -l` pads its count on macOS, so `tail -n +$n0` needs `$((n0+0))` or an arithmetic expansion.
- `api:` reboots a board after 15 min without a client: `reboot_timeout: 0s` on `api:` and `mqtt:` for
soak tests.
@@ -5,15 +5,13 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
## ESP8266 (D1 mini, NodeMCU, CH340 or CP2102)
- Opening the port usually resets the board through DTR, but not always, and on a CH340 even with `--released`, so
the first boot (or `Connection lost`) in a capture can be that reset; `serial_capture.py --reset` pulses RTS to
capture from boot. 74880 baud shows the ROM boot messages (`rst cause`, `boot mode`).
- Opening the port usually resets the board through DTR, but not always; `serial_capture.py --reset`
pulses RTS to capture from boot. 74880 baud shows the ROM boot messages (`rst cause`, `boot mode`).
- UART0 can be looped back onto itself (`USC0(0) |= (1 << UCLBE)` in an `on_boot` lambda) so a UART
component is testable without its device: frames written with `uart.write` arrive on RX. Log over the
API with `logger: baud_rate: 0`, since the serial port carries the component's bytes.
- `esphome logs` on some CH340s emits thousands of duplicated garbage lines right after the RTS reset;
a plain pyserial capture at 115200 is reliable. A CH340 can also give garbage at 921600 baud;
`logger: baud_rate: 460800` with `serial_capture.py --baud 460800` carries long `dump: raw` frames cleanly.
a plain pyserial capture at 115200 is reliable.
- A failed esptool write can wedge the adapter (`termios` error 22, or an open that hangs) until replug.
- The preference sector survives a reflash: a provisioned API key or saved Wi-Fi credentials stay
active. `esptool erase_flash` gives a clean device.
@@ -26,18 +24,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
row in the PR.
- AsyncMqttClient drops inbound topics over 128 characters; topics of 255 or more wrap a counter and
stall the link until the broker drops TCP.
- `Arduino.h` already declares `_heap_start`; a lambda can use `&_heap_start` directly, and a second `extern` fails.
- `Hardware Watchdog` with `wDev_ProcessFiq` in the stack means interrupts were held off too long for the Wi-Fi
interrupt handler; `Software Watchdog` with `pm_send_nullfunc` means the loop was busy too long. Code that takes
interrupt locks needs a load run to show either (`scripts/esp8266_lock_load/`, see recipes/one_wire.md).
- `GPI` bits 16 to 31 are strapping bits (mask with 0xFFFF). GPIO16 has no interrupt; read `GP16O` for an output.
- Cover positions and other restored entity state live in RTC memory by default, so they survive `erase_flash`
and a dev build's state carries into the next build's first boot. Set `esp8266: restore_from_flash: true` on A/B
configs that erase between builds.
- The ROM `rijndaelKeySetupEnc` at 0x40009300 builds a correct 128-bit schedule (FIPS-197 A.1 words match) but does not
return the round count, so the SDK's `aes_encrypt` loops on garbage rounds and the board Soft WDT resets in
`rijndaelEncrypt`. Do not `--defsym` it over the libcrypto copy; a wrapper in the SDK signature (returns -1 unless
128 bits, calls the ROM, returns 10) passed the FIPS vectors and a WPA2 soft AP join.
## ESP32 classic devkit
@@ -54,13 +40,10 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
- Signed OTA testing: build with `esp32.framework.advanced.signed_ota_verification.verification_keys`
under a different name, sign with `python -m espsecure sign_data --version 2 --keyfile
tests/components/esp32/dummy_signing_key.pem`, upload with `esphome upload --file`. The verifier
refuses unsigned images afterwards, so restore with a signed plain build. The full accept and reject case set
(several keys, CRC-only edits) is in `recipes/ota.md`.
refuses unsigned images afterwards, so restore with a signed plain build.
- An ESP32 whose bootloader has no app rollback is protected only by `safe_mode` after a bad OTA. Before sending a
change to such a device that nobody can reach by hand, boot the same code on a devkit first and keep a fallback
image of the merge base built.
- On modules where GPIO16 and GPIO17 are taken by the module itself (PSRAM on WROVER modules, for example), a
`uart:` bus on those pins boot-loops with a watchdog reset during setup. Use GPIO21 and GPIO22.
## ESP32-S3, C3, C6, H2 with native USB
@@ -69,7 +52,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
`Disabling RNG early entropy source`, then 0 bytes, esptool reports `No serial data received`). Only a
USB replug fixes it. Fallback: `esphome logs --device <name>.local` (API log stream, which drops lines
while Wi-Fi is off).
- The USB console drops a line now and then even when it stays up; do not count events by log lines alone.
- The ESP32-P4 before silicon revision 3 is bricked by rev3-only firmware (boot loop at bootloader entry). The
Waveshare ESP32-P4-WIFI6 rev1.3 board is such pre-rev3 silicon.
- S3, C3 and C6 all enumerate as `303a:1001`; confirm the chip with `esptool --port <port> chip_id` before flashing.
@@ -83,27 +65,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
- A board left in the ROM bootloader after an upload (BOOT held, or the USB JTAG reset landing in download mode
again) starts its app without a replug through `esptool --port <port> --after watchdog-reset chip-id`; flash a
minimal Wi-Fi plus API image first when recovering from a crash-looping one.
- On macOS a software reboot of an ESP32-C3 does not drop its USB JTAG serial port: pyserial keeps reading and gets
no error, so a client that watches for a port reset misses it. Count ESPHome's `Running through setup()` line
instead.
- W5500 SPI Ethernet with IPv6 on (seen on an ESP32-S3, #22676): every `safe_reboot()` (OTA, restart button)
aborts while an API client is subscribed to logs. `mdns_free()` leaves the IPv6 group inside `tcpip_api_call`, the
W5500 driver logs "IPv6 multicast is always filtered in by W5500", the logger sends that line to the API inline,
and the nested `LwIPLock` in `setsockopt(TCP_NODELAY)` releases the core lock it did not take, so `udp_recv`
asserts. No crash with IPv6 off, with no log subscriber (a connected client that does not subscribe is fine), or
on RMII Ethernet. Reading the crash record afterwards: `recipes/crash_report.md`.
- `usb_serial_jtag` holds a `NO_LIGHT_SLEEP` power lock by itself, so a light sleep soak on a native USB board logs
to UART0 instead (see `recipes/light_sleep_gpio_wake.md`).
- The Waveshare ESP32-P4-WIFI6 needs `engineering_sample: true`. Its Wi-Fi is an ESP32-C6 over SDIO through
`esp32_hosted`; a test with no network logs over UART0 to the CH343 console.
- ESP32-S3: GPIO43/44 cannot be used to cross-wire two UARTs: the UART0 IOMUX TX is lost when another UART takes the
pad as RX. TX17/RX16 and TX16/RX17 work. A task woken by `vTaskDelay` runs about 0.5 ms after the tick, on dev too.
## ESP32-C2 (ESP8684, 26 MHz crystal)
- Boards with a 26 MHz crystal print the ROM banner (`ESP-ROM:esp8684`) at 74880 baud; at 115200 it is garbage.
- ESP-IDF builds need `esp32: framework: sdkconfig_options: {CONFIG_XTAL_FREQ_26: y}`, or the app console runs at
the wrong baud rate (the upload log warns `Crystal frequency mismatch`).
## RP2040 / RP2350 (Pico W, Pico 2 W)
@@ -131,27 +92,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
each RD_REQ with one byte from a 256-byte register file, so any read length works (used for BME280/BMP280 in
esphome#22236). The ESP32-S3 I2C slave v2 driver is not usable for register emulation: answering from a task or
straight from the interrupt both gave shifted multi-byte reads.
- RP2350 PIO instruction memory survives a reflash (the 1200 baud touch and the UF2 reboot), so a program loaded by
the previous firmware can hide a missing `pio_add_program`. Clear it from an `on_boot` priority 1000 lambda
(`pio1->instr_mem[i] = 0` for i < 32) to start from power-on state; `scripts/pio_probe/pio-common.yaml` does.
- `RP2Preferences::reset()` erases the flash sector but not the RAM copy the preferences load from, so the LibreTiny
`on_boot` wipe-once pattern still boots with the old saved Wi-Fi credentials. Use a fallback AP config with
`networks:` and a fresh `name:` per build instead.
- WIZnet W5100S-EVB-Pico (RP2040, `rp2: board: wiznet_5100s_evb_pico`): W5100S on SPI with CLK 18, MOSI 19, MISO 16,
CS 17, INT 21, RST 20 (`ethernet: type: W5100`). ESPHome firmware gives it no USB CDC port; reach it over Ethernet,
OTA by address, and flash it over USB only through BOOTSEL and the UF2 drive. Keep `interrupt_pin` in every build:
without it OTA starves and never completes (`recipes/arduino_pico.md`), which strands the board until a USB flash.
The W5100S answers ICMP in hardware, so a ping reply says nothing about the RP2040; probe TCP instead (OTA port
2040, API 6053, HTTP 80). In safe mode only 2040 answers.
- Through a Pico W `bluetooth_proxy`, about 1 to 2 in 5 BLE connects fail with 0x3E (`connect err=62` in the
client) on every build; retry.
- arduino-pico framework bumps and the RPAsyncTCP SSE stall: see `recipes/arduino_pico.md`.
- USB CDC sends nothing until the host asserts DTR; a client that opens with DTR low (`improvcli.py` without
`--keep-dtr`) sees silence.
- Powering a breakout from the Pico: use 3V3(OUT), pin 36. Pin 37 is 3V3_EN, which switches the Pico off when pulled
low.
- W5100S-EVB-Pico with its USB attached shows a CDC port (`2e8a:1027`) while it runs ESPHome, so `esphome upload
--device <that port>` (picotool) flashes it with or without `interrupt_pin`.
## BW15 kit (RTL8720CF, LibreTiny rtl87xx)
@@ -193,14 +133,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
- Wrong-MD5 OTA test without changing firmware: monkeypatch `espota2.hashlib.md5` so only the image checksum (data
over 64 KB) returns a wrong hexdigest. Give each build its own `name:` and read `device_info().name` after a
restart to see which image booted.
- Logger UART: `DEFAULT` is UART2 (the kit's USB serial). `hardware_uart: UART1` hangs after the boot banner and
never joins Wi-Fi, on dev too. `UART0` moves both ESPHome and `lt_log` output off the kit's port. LibreTiny's own
`LT_I` lines need `framework: loglevel: info` (the default is warn).
- A config with no Wi-Fi network and no `ap:` resets about 10 s into setup when it has `web_server:` (dev too):
`web_server` setup blocks with no interface up, and no crash record is left. VERBOSE logs show the last
`Setup <component>` line.
- A BW15 kit whose CH340 drops off the bus at "Initializing WIFI" on every boot (a boot loop on any image) is short of
USB power: use another hub port or a powered hub. Read the results over API logs between resets meanwhile.
## BK7238, WB3S, CB3S (BK7231N) (LibreTiny bk72xx, CH340, RTS on CEN)
@@ -215,8 +147,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
assuming either.
- `flash_bk72xx.py` misses the link now and then (`Timed out attempting to link with chip`); retry up to 3
times before suspecting the board.
- Both flash wrappers pass ltchiptool `-L` with the newest installed LibreTiny platform (`scripts/lt_platform.py`):
its default `~/.platformio/platforms/libretiny` can be an older copy that rejects BK7238 UF2s as unsupported.
- Crashes: `__builtin_trap()` and a call into unmapped memory both end as an undefined-instruction trap
(`bk_misc_get_start_type()` 6, `lt_get_reboot_reason()` Crash); reads, unaligned loads and stores to
unmapped addresses never fault on the ARM968. Without Wi-Fi the SDK trap handler spins with interrupts off
@@ -252,9 +182,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
... !=`) and the old image stays.
- Power save: on the BK7238, `power_save_mode: light` with LibreTiny 1.14.0 gives ping times of 3 to 6 s because
the BDK sets a listen interval of 30 beacons when `CFG_LOW_VOLTAGE_PS` is 1 (esphome#19491).
- LibreTiny's builder adds `-D'delay(ms)=delayMilliseconds(ms)'`, so `esphome::delay` is really
`esphome::delayMilliseconds` on BK72xx and a YAML lambda calling `delay()` fails to compile on dev. In a header use
`(esphome::micros)()` to dodge the `main.cpp` function-like macros.
## BW12 and other AmebaZ boards (RTL8710BX, LibreTiny rtl87xx)
@@ -299,9 +226,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
- A 4-way handshake that never finishes after association is reported as `RTW_WRONG_PASSWORD` on AmebaZ
even with the right password; AmebaZ2 reports a real wrong password as a 4-way handshake timeout.
- The driver also posts a SCAN_DONE (number 0) inside every join, on AmebaZ2 too.
- RTL8710B GPIO interrupts fire on one edge only, and LibreTiny maps CHANGE to rising. The Arduino `digitalRead` called
from an ISR switches the pin back to GPIO and frees its interrupt (esphome#22400, #22403).
- The FreeRTOS tick is 1/1024 s, so `millis()` on the tick ran 2.4 % fast (esphome#22404).
## BW16 kit (RTL8720DN, AmebaD)
@@ -329,10 +253,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
partition even when the upgrade state is still `UPG_STATE_DOWNLOAD_ING` (esphome#22111).
- Without serial access to recover, never send a corrupted image; wrong-MD5 and cut-upload tests with valid
images are safe.
- On the WL2H-U `logger: hardware_uart: DEFAULT` resolves to UART1, LibreTiny's log port for that board.
- Over a poor link (about 60 ms ping and 5 % loss) an ESPHome OTA takes anywhere from 50 to 445 s and a web `/update`
of a UF2 can abort early while ESPHome OTA still works. Treat such a board's results as crash or no crash only, and
settle questions about rates, heap after streams or OTA aborts on a BK7238 (same LibreTiny AsyncTCP).
## nRF52840
@@ -362,10 +282,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
the port open, and the logger warns `logger took a long time`. Print one line per `loop()` instead.
- Zephyr settings survive a reflash: key a `restore_value` global by a `run` substitution when a test needs
a never-saved value.
- `2fe3:0100` is Zephyr's default USB VID:PID, so `ports.py` labels any board running a Zephyr ESPHome build as an
nRF52840 Dongle. Tell an ItsyBitsy from a PCA10059 with a 1200 baud touch: the ItsyBitsy shows its `ITSY840BOOT`
UF2 volume, the PCA10059 its Nordic DFU port.
- nRF52 Zephyr builds use newlib; the nRF54LM20 builds use picolibc.
- Thread: nRF52 `openthread:` accepts only `tlv:`; the individual keys (network_key, channel, ...) are rejected at
validation. To list nearby networks with channel and PAN ID, call
`otThreadDiscover(InstanceLock::acquire().get_instance(), 0, 0xFFFF, false, false, cb, nullptr)` from an
@@ -419,16 +335,6 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
- BDK headers such as `power_save_pub.h` have no `extern "C"`; wrap them when including them from the Arduino
core's C++ files.
## Shared PlatformIO directory
- The stamp file is `.esphome.pio.stamp.json`; a mismatch deletes `packages`, `platforms` and `.cache`.
- Building an old release (2025.7.0, Arduino ESP32) against the shared directory swapped `tool-scons` for an older
version and broke it midway (`No module named 'SCons.SConf'`) for every PlatformIO build until a newer build
reinstalled it. Build old releases with their own `PLATFORMIO_CORE_DIR`.
- Two arduino-pico framework versions share the plain `framework-arduinopico` package directory, so an A/B swaps the
framework on every build; give each side its own `PLATFORMIO_CORE_DIR`.
- The general rule (a private directory whenever sessions may use different Pythons) is ground rule 8 in `SKILL.md`.
## LibreTiny crash and abort() facts (esphome#21596, #22288)
- `abort()` on dev: the BK7238 hangs for good (the main loop stops and the watchdog never fires once Wi-Fi is up;
@@ -508,45 +414,3 @@ Ports are not listed on purpose: they follow the USB hub port and change on ever
- LED0 P1.22, LED1 P1.25; BUTTON0 P1.26, BUTTON1 P1.09, BUTTON2 P1.08, BUTTON3 P0.05 (active low, pull-up). GPIO
P1 registers at 0x500D8200 (IN +0x00C, PIN_CNF +0x080), P0 at 0x5010A200.
- Binary sensor state changes are not logged at DEBUG; log the states or count `on_press` to see button presses.
- FICR INFO.PART reads 0x054BC20B; the build target is `nrf54lm20dk`.
- Zephyr `micros()` on nRF has 30.5 us resolution. Time short blocks with the DWT cycle counter instead; nRF54L idle
clears `DWT_CTRL`, so enable it again before each timed block.
- The nrf52 logger defaults to USB_CDC (the chip's own USB); set `logger: hardware_uart: UART0` to get the log on the
J-Link VCOM port.
- I2C (#22642): TWIM21 only takes pins on P1 or P3; P0 pins are rejected at config. BUTTON1 (P1.09) and BUTTON2
(P1.08) pads work as SDA and SCL. A bus with no pull-ups times out (-116) on every scan address, so setup takes
about 56 s. The JLinkExe `st` command reads VTref to check the I/O voltage.
## Seeed XIAO nRF54LM20A
- CMSIS-DAP probe (`2886:0068`) with one serial port that carries the console (uart20 on P1.11 and P1.10, #22599).
It ships with a Seeed demo.
- Flash with `esphome run` (pyOCD, MCUboot plus the signed app). A plain `esphome upload` flashed only the bare app
on #22599, and the board stayed dark.
- Reset: `<sdk-nrf python> -m pyocd reset -t nrf54lm20a -u <probe id>`.
- nPM1300 PMIC at 0x6B on P1.18 (SDA) and P1.17 (SCL) (#22661; `i2c:` builds on i2c21 at 100 kHz). Per the upstream
Zephyr board files, BUCK2 supplies VSYS_3V3, the nRF54 and the SAMD11 USB debugger: never switch it off (there is no
way back without a power cycle, and the probe dies with it). LDO1 feeds the IMU and microphone (3.3 V, LDO mode) and
is safe to toggle. BUCK1 and LDO2 are not described anywhere; leave them alone.
- nPM1300 power-on state: BUCK2 on at 3.3 V, BUCK1 off, LDO1 and LDO2 off in load switch mode (enabling one as is
passes VINLDO straight through), charger off, VBUS limit 100 mA, NTC select 10 k. Flows: `recipes/npm1300.md`.
## I2C and SPI breakouts
- GPIO switches on an I2C expander (XL9535 and others) write `value_or(false)` at setup even with
`restore_mode: DISABLED`, so test pin retention on a pin no switch uses.
- ADS1015: `sample_rate: 8` maps to 128 SPS at `12_BITS`.
- A bus held low (`Performing bus recovery`, no devices found) on a new breakout can be a bad header joint; reflow
it before blaming the driver.
## Harness tricks and old releases
- Protected members and methods can be reached from a test header listed in `esphome: includes:` with the explicit
instantiation access trick (`template struct Rob<Tag, &C::member>;`), even on `final` classes; declare the robbers at
global scope.
- API driven light effects that write template outputs every loop (about 5k writes/s) can lag API commands by seconds:
wait for the API state to show each command before reading results.
- Building ESPHome 2025.10.x (pioarduino ESP-IDF 5.4.2) from a path under `/tmp` on macOS fails with "Multiple ways to
build the same target" (the /tmp to /private/tmp alias); put the yaml, build and PLATFORMIO_CORE_DIR elsewhere. Its
IDF penv under Python 3.14 lacks the tools: install `-r packages/framework-espidf/tools/requirements/requirements.core.txt`,
then pin `idf-component-manager~=2.1.0 esp-idf-kconfig~=2.5.0` (newer ones reject interface_version 3).
@@ -4,7 +4,7 @@ Each flow names the pull requests it was used on, so their result tables show wh
One file per component, named after it:
- [`api.md`](api.md): noise handshake and OTA encryption on the ESP8266, outgoing connection wake timing on deep sleep and after a Wi-Fi drop, user defined action listing and execution with a raw plaintext client, store_yaml recovery over Noise or plaintext, rapid connect/disconnect stress (`connstress.py`)
- [`api.md`](api.md): noise handshake and OTA encryption on the ESP8266, outgoing connection wake timing on deep sleep and after a Wi-Fi drop, user defined action listing and execution with a raw plaintext client, store_yaml recovery over Noise or plaintext
- [`ble_midi.md`](ble_midi.md): BLE MIDI client against a fake ESP32 instrument, every packet shape and the send actions
- [`captive_portal.md`](captive_portal.md): fallback AP and /wifisave hammer
- [`cse7766.md`](cse7766.md): Sonoff S31 parser check over the API
@@ -14,31 +14,28 @@ One file per component, named after it:
- [`ld2420.md`](ld2420.md): hub setup and sensor states with a simulated module on a UART0 loopback (energy and simple mode)
- [`ld2450.md`](ld2450.md): polygon zones, zone number round trip through the radar's readback, ESP8266 loopback harness without a radar, presence hold and link loss on a frame stream, BT MAC readback
- [`ld2412.md`](ld2412.md): output pin level through the basic configuration command, held back writes
- [`libretiny.md`](libretiny.md): rtc preference emulation, BK72xx and RTL8720C crash report on the next boot with a ROM dump matcher, BK7238 BLE bond store safety with a full flash backup and per partition diff, BK7238 OTA versus the board layout and bootloader, pulse counting, RTL8710B edge interrupts and millis timing
- [`modem.md`](modem.md): PPPoS modem component boot check with no modem attached, loop blocking and disable timing, SIM7080G behaviour with no SIM and with no data plan through a raw UART passthrough
- [`libretiny.md`](libretiny.md): rtc preference emulation, BK72xx and RTL8720C crash report on the next boot with a ROM dump matcher, BK7238 BLE bond store safety with a full flash backup and per partition diff, BK7238 OTA versus the board layout and bootloader
- [`modem.md`](modem.md): PPPoS modem component boot check with no modem attached, loop blocking and disable timing
- [`mqtt.md`](mqtt.md): broker bench, QoS, reconnect, crash log over MQTT, WebSocket transports with a local and a public broker, TLS trust sources (own CA, client certificate, certificate bundle, no verification) with the CLI check, MAC suffix topics and will messages on two boards, shutdown message delivery on restart, disable and deep sleep
- [`nrf52.md`](nrf52.md): dongle board support, zephyr_ble_server connection slot under reordered callbacks, SMP over UDP OTA over Thread with reset faults, MCUboot swap limit and stuck slot recovery, TCP stack probe, sdk-nrf 3.4.1 build and USB notes
- [`npm1300.md`](npm1300.md): nPM1300 PMIC on the XIAO nRF54LM20A: probe build, every entity with a register dump and regulator toggles, power-on state and no-battery readings
- [`nrf52.md`](nrf52.md): dongle board support, zephyr_ble_server connection slot under reordered callbacks, SMP over UDP OTA over Thread with reset faults, MCUboot swap limit and stuck slot recovery, TCP stack probe
- [`online_image.md`](online_image.md): stalled and malformed downloads, PNG, JPEG, BMP and QOI decode time per loop pass with large images and small buffers
- [`openthread.md`](openthread.md): border router on the S3 host plus H2 RCP
- [`ota.md`](ota.md): upload without a configuration and the refusal checks, overlap and lock, timeouts, deflate, upload timing A/B per build and board with deflate and raw, on-device inflate benchmark, OTA automations on two platforms
- [`rtsp_server.md`](rtsp_server.md): camera streaming on the Waveshare ESP32-S3-ETH
- [`scheduler.md`](scheduler.md): loop wake A/B, interval startup delay and control from lambdas, RP2 scheduler lock under IRQ and lwIP load
- [`scheduler.md`](scheduler.md): loop wake A/B, interval startup delay and control from lambdas
- [`udp.md`](udp.md): datagrams, multicast, sockets, `udp.write` payload shapes
- [`usb_cdc_acm.md`](usb_cdc_acm.md): nRF52840 dongle host driver
- [`wifi.md`](wifi.md): reboot_timeout with a fallback AP, DHCP lease cache, Easy Connect (DPP), WISP bridge with an ethernet upstream, reboots against a WPA2/WPA3 PMF access point, WISP and ALWAYS_ON AP switching, AP restarts with WPA3 mixed and LIGHT power save, soft AP clients with the BLE tracker scanning, ESP8266 power save after captive portal provisioning
- [`wifi.md`](wifi.md): reboot_timeout with a fallback AP, DHCP lease cache, Easy Connect (DPP), WISP bridge with an ethernet upstream, reboots against a WPA2/WPA3 PMF access point, WISP and ALWAYS_ON AP switching, AP restarts with WPA3 mixed and LIGHT power save
- [`zephyr.md`](zephyr.md): preferences on the nRF52840 dongle
- [`zephyr_coredump.md`](zephyr_coredump.md): crash and decode on nRF52 boards
- [`zwave_proxy.md`](zwave_proxy.md): frame checksums on a live proxy
- [`one_wire.md`](one_wire.md): ESP8266 interrupt locks under Wi-Fi load with a DS18B20 search and a busy-wait control
- [`cc1101.md`](cc1101.md): sleep and wake checks read directly from the chip (CHIP_RDYn, PATABLE loss, registers per setter)
- [`globals.md`](globals.md): restoring globals with storage rtc and flash across every reset kind
- [`homeassistant.md`](homeassistant.md): light platform mirrored from a local Home Assistant, held reports versus later calls, wizard-linked entities on every platform, homeassistant switch on a script with a host build
- [`homeassistant.md`](homeassistant.md): light platform mirrored from a local Home Assistant, held reports versus later calls
- [`usb_audio.md`](usb_audio.md): USB audio speaker and microphone, the usb_host FIFO split, and Music Assistant playback through sendspin
- [`dfrobot_sen0395.md`](dfrobot_sen0395.md): SEN0395 mmWave command flow, module quirks and overlong lines through its error reply
- [`speaker.md`](speaker.md): media player pipeline tasks on the Connect AUX-2 with the `speaker` platform player
- [`web_server.md`](web_server.md): state SSE stress and the HTTPD task stack on web_server_idf, entity icon and device class readback, per-session `/events` listing, `/events` head, stall, Wi-Fi drop and churn checks
- [`uart_sim.md`](uart_sim.md): UART sensors simulated on a classic ESP32 with loopback and cross-wired UARTs, a Duco ventilation box model on a second UART
- [`web_server.md`](web_server.md): state SSE stress and the HTTPD task stack on web_server_idf, entity icon and device class readback
- [`uart_sim.md`](uart_sim.md): UART sensors simulated on a classic ESP32 with loopback and cross-wired UARTs
- [`gpio_expander.md`](gpio_expander.md): interrupt driven expander inputs, loop idle checks, testing a device out of reach
- [`led_strip.md`](led_strip.md): forcing a failed strip allocation, RMT symbols per chip, loop cost per strip
- [`remote_base.md`](remote_base.md): remote_receiver protocol decoding through a one-pin loopback on ESP32 and ESP8266, no IR hardware
@@ -64,28 +61,12 @@ One file per component, named after it:
- [`rs485_frame.md`](rs485_frame.md): DLE/STX/ETX bus on an ESP32-C3 with a cross-wired simulated bus device (frame cases, gate timing)
- [`hx711.md`](hx711.md): fake HX711 driven from the other core, every published value matched to the served conversion
- [`automation_trace.md`](automation_trace.md): automation codegen A/B, a marker trace of chains, conditions, loops and script modes over the API
- [`bluetooth_proxy.md`](bluetooth_proxy.md): held GATT link after a half-open API session, frozen Home Assistant stand-in, out-of-order GATT discovery responses injected on the proxy, GATT UUID forms (16 bit, 32-bit base, 128 bit) compared against an ESP32 proxy
- [`bluetooth_proxy.md`](bluetooth_proxy.md): held GATT link after a half-open API session, frozen Home Assistant stand-in
- [`crash_report.md`](crash_report.md): crash buttons, fetching the previous boot's crash report over the API, per-chip abort facts
- [`https_idf.md`](https_idf.md): web_server_idf HTTPS test component, listener switch, stalled TLS clients, bad key
- [`mdns.md`](mdns.md): legacy unicast mDNS dump, hammer and latency straight to the device
- [`sim_buses.md`](sim_buses.md): PCF8574/HD44780, MAX7219, SPI spy, packet_interface and SN74HC165 simulators, status LED probe, fake Home Assistant, cg.static_buffer probes, a TRF7962A NFC reader chip behind a fake SPI bus
- [`sim_buses.md`](sim_buses.md): PCF8574/HD44780, MAX7219, SPI spy, packet_interface and SN74HC165 simulators, status LED probe, fake Home Assistant, cg.static_buffer probes
- [`sigma_delta.md`](sigma_delta.md): pulse density of the sigma_delta speaker pin read inside the ESP32-P4, rtttl note onset and release
- [`microphone_http_stream.md`](microphone_http_stream.md): raw PCM over HTTP with a counter fake mic or a USB mic, multi-reader client
- [`i2s_audio.md`](i2s_audio.md): I2S microphone fed by an I2S TX channel on the same S3 through three jumpers
- [`voice_assistant.md`](voice_assistant.md): late TTS start on the Voice PE with a slow announcement URL
- [`mcp251xfd.md`](mcp251xfd.md): MCP2518FD CAN FD over SPI in loopback, forced modes, chip reset and bus-off recovery, TX objects read back from chip RAM, idle SPI cost from a PCNT on CS
- [`crc32.md`](crc32.md): crc32 vectors on every platform path and the signed OTA signature block CRC on an ESP32-C3
- [`i2c_isr_fault.md`](i2c_isr_fault.md): software INA219 on the master's own pads with seeded bus faults, interrupt storm and stuck bus
- [`bmp390.md`](bmp390.md): BMP390/BMP388 register readback, INT data ready rate, start write faults by software NACK or held SDA, and the IDF I2C hang after a held SDA
- [`ipv6.md`](ipv6.md): IPv6 link-local and SLAAC boot and reconnect timing on ESP32 Wi-Fi, with an injected esp-netif delay that forces the race
- [`mcp23017.md`](mcp23017.md): MCP23017 IOCON aliasing and INT line behavior (open drain, push-pull, polling) on an ESP8266 with PA to PB jumpers
- [`pwm_capture.md`](pwm_capture.md): every LEDC PWM period captured on the same ESP32-C6 through RMT receive and PCNT, with and without power save
- [`spi.md`](spi.md): SPI bus capture on the same board with no chips (ESP32 SPI3 slave, edge interrupt snapshots on C3, ESP8266 and Pico 2 W), checked and diffed between builds
- [`heap_alloc.md`](heap_alloc.md): operator new counting, and a ring of allocation callers decoded with addr2line to tell the first run after boot from repeats
- [`wled.md`](wled.md): WLED realtime sender on a board's own access point
- [`combination.md`](combination.md): combination sensor NaN behaviour from the last source
- [`improv.md`](improv.md): Improv serial with stray bytes, scan, bad and good passwords from a client that reads frames out of the log stream, Improv over BLE, per board notes
- [`arduino_pico.md`](arduino_pico.md): arduino-pico framework bump A/B: W5100S Ethernet soak, Pico W fallback AP client count, Pico W bluetooth_proxy GATT hold, web_server SSE stalls, `/events` churn crashes and never completed connects, W5100S OTA without `interrupt_pin`
- [`bluedroid_gattc_queue.md`](bluedroid_gattc_queue.md): queued MTU request and execute write recursion that overflows BTU_TASK, ble_client central against an ESP32-S3 peer
- [`serial_proxy.md`](serial_proxy.md): simulated serial device on a cross-wired UART, byte-exact client with read pauses, ASH tap ACK timing
- [`network_dns.md`](network_dns.md): network.priority Ethernet plus Wi-Fi losing lwIP DNS servers (#19805)
@@ -56,46 +56,3 @@ include of random base64 lines with `include_secrets: true` gives four chunks (1
0bd3738430: `include_secrets: true` fails the build when a package in another directory uses `!secret` and has no
`secrets.yaml` beside it; a copy next to the package works around it.
Results: https://github.com/esphome/esphome/pull/16445#issuecomment-6335435397
## Allocations of a connecting client (ratgdo/esphome-ratgdo#705)
An API client that connects allocates about 15 times on ESP32: the connection object, then the list entities and
initial state batches growing the `DeferredBatch` vector. A log subscription adds more per line. Connect before a
heap measurement and subscribe to logs only afterwards (`heap_alloc.md`).
## Native API connect/disconnect stress (#14679, #14687, #14706)
`scripts/api/connstress.py HOST [ITERATIONS] [CONCURRENCY]` (aioesphomeapi; `API_KEY` env for a noise key) opens and
closes native API connections as fast as it can, 500 attempts with 4 at a time by default, and ends with
`RESULT PASS` or `RESULT FAIL`. It is the test that found the raw lwIP socket heap corruption, so run it for any change to
the socket layer, the scheduler or locking on RP2040/RP2350 and ESP8266, with a bounded serial capture on the device:
the client only sees resets, the crash is in the log.
- RP2040/RP2350 before #14687: crashed after 3 or 4 iterations (`malloc` from the lwIP accept IRQ while the main loop
was in `malloc`: double free in `~APIFrameHelper`, `_malloc_r` in `APIConnection()`); after: 500/500. #14679 first
added the lwIP locking between the lwIP callbacks and the main loop on RP2040.
- ESP8266 before #14706: crashed within about 50 iterations (`umm_malloc_core` LoadStoreError, use after free of a
closed PCB's callbacks); after: 490/500, the 10 failures are connection resets from overload, which is expected there.
- ESP32 (BSD sockets) passes 500/500 on both and is the control.
- Compare against dev on the same board; count reboots and decode crash addresses as well as the client's failures.
## API overflow buffer driven and counted on the device (#22771)
`scripts/api/overflow/`: `ovfwrap.hpp` wraps `APIOverflowBuffer::enqueue_iov` and `try_drain` with `-Wl,--wrap` and
counts calls and bytes; the `try_drain` symbol differs per platform (`...6socket11LWIPRawImplE` on ESP8266,
`...6socket13BSDSocketImplE` on ESP32 IDF), so pass it as a substitution and check `nm` shows both `__wrap_` symbols
(a wrong name silently drops the wrapper). ESP-IDF builds keep `nm` in the IDF tools cache
(`xtensa-esp-elf/*/xtensa-esp-elf/bin/`), not under the PlatformIO directory. `putbe-common.yaml` (60 sensors from `putbe-sensors.yaml`, which `gen_sensors.py` writes; four
800 char text sensors flooded every 50 ms, counters as sensors, plus two udp `entity_transport` links) and
`apitest.py HOST OUT` (2 KB receive buffer, 3 x 6 s `pause_reading`, validates every burst state and the final states;
`API_KEY` in the env). Result on #22771: 82 to 159 enqueues per run on a D1 mini and an ESP32, no drops.
`scripts/entity_transport/trcheck.py` compares remote entity values with the sender's published ones over the API on
both boards; it accepts late updates and checks floats by the formula the sender uses, so only values the sender never
published fail. Declare remote entities for both the base and the PR name of the peer so one image serves all four
pairings. Facts from that run:
- Xtensa GCC (lx106 and esp32) defines `__CHAR_UNSIGNED__`; macOS host builds use signed `char`, so a host harness
can miss a sign bug in byte handling.
- udp `packet_interface` with broadcast between a D1 mini and an ESP32 loses most hdlc acks; unicast `addresses:` still
resets the reliability session a few times a minute (dev and PR alike). A D1 mini's API can deliver states 5 to 11 s
late under that load.
@@ -1,42 +0,0 @@
# arduino-pico framework bumps (RP2040/RP2350)
Scripts: `scripts/rp2_web_ble/` (README there). Used for #22671 (6.1.1 to 6.3.2).
- Build PR and dev with a private `PLATFORMIO_CORE_DIR` (`scripts/pio_build.sh` does; see `reference/boards.md`).
Check the compile log line `framework-arduinopico @ 1.60302.0` per build.
- Wired Ethernet (W5100S EVB): 6.3.2 defines `LWIP_ASSERT_CORE_LOCKED()` as `async_context_lock_check` on the
Ethernet context (`cores/rp2040/lwip_wrap.cpp`), so unlocked lwIP calls panic there. Soak the API (200 ms template
counter, 1 s uptime, logs subscribed), load pages, OTA and reboot; the rp2 crash handler shows a panic as
`Crash (HardFault)` in the debug reset reason on the next boot.
- Fallback AP client count: `ap_clients_` is protected and `WiFiComponent` is final; read it from an
`esphome: includes:` header through an explicitly instantiated member pointer (`appeek.hpp`). Join raises it to 1.
- bluetooth_proxy on the Pico W: `rp2_ble_tracker:` + `bluetooth_proxy: active: true`. Drive with `gatthold.py`
against the ESP32-S3 server in `scripts/ble_notify/server.yaml`. UUID forms and the 0x3E connect failures are in
`bluetooth_proxy.md`. A "Failed to send notify data response" warning at each disconnect is normal.
- web_server SSE on the W5100S (dev 2a6a1a050e4, 6.1.1): every `/events` stream stops after 0.3 to 11 s (alone, with
the API, or with page loads alike), then the EVB closes it with a FIN 5.8 to 6.3 s after its last data segment, so
curl exits 0. The pcap shows every byte ACKed by the host, window 65535, no retransmits, nothing sent in the gap:
the device simply stops writing. The 5 s gap matches RPAsyncTCP `ASYNC_MAX_ACK_TIME` (likely `_pcb_busy` not
cleared, then `_onTimeout` closes). A new `/events`, GET / and the API keep working. Pico W: 60 s steady. On macOS
`tcpdump` runs without sudo once your account is in the bpf group. Tools: `sserun.sh HOST TAG SECS alone|api|pages
N` (pcap, netstat poll, API VERBOSE log, probes) and `pcapsum.py PCAP DEVICE_IP` (per connection last data, FIN/RST,
retx). Confirmed with counters patched into the build's `.piolibdeps` copy of RPAsyncTCP: every stall is one
`_sent()` underflow with `_tx_unacked_len` 0 and the pcb idle after `send()`; ESPAsyncWebServer 3.12.1 does not
help. The esphome-libs RPAsyncTCP fork (`esphome-1.3.2`, 8efb83c) streams 16 x 30 s with no stall. `stallctr.py PCAP
IP APILOG` lines the counters up with each stall.
- `H=<device ip> churn.sh TAG SECS` (4 loops of 0.2 to 1.5 s `/events` connections) crashes the rp2 web_server: dev on
the EVB, the fork build on the EVB and the Pico W. HardFault in `AsyncEventSource::send` from
`deferrable_send_state`, about one reboot every 10 to 20 s. Ten quick reboots put the board in safe mode (ping and
OTA port 2040 only); OTA a build to recover.
- **Never flash the W5100S EVB a build without `interrupt_pin`** unless you can reach BOOTSEL: arduino-pico then polls
the W5100 from an async_context timer that every lwIP call pushes back, so the esphome OTA read loop starves the
poller and OTA never completes (the device ACKs the hello, then never the features byte). Web and API still work.
The board has no USB CDC port with ESPHome firmware, so the only way back is BOOTSEL and the UF2 drive.
- `churnflow.sh` (in `rp2_web_ble/`) repeats `churn.sh` with heap and main loop checks and reflashes by OTA when a
round left the board in safe mode.
- `leakflow.sh` with `neverconn.py HOST N rst|head`: ESPAsyncWebServer 3.9.6 leaks about 175 B per `/events` client
that resets before the response head is ACKed (52 KB per 300); 3.12.1 stays flat.
- `/events` churn also hangs the BK7238 main loop or drops it off the network on dev; recover with a serial reset.
- The API has 3 sockets on these builds: connstress at concurrency 4 always shows some "dropped after hello" failures;
use 2 for a clean 500/500.
- IRQ context scheduler test for #22688: `scripts/rp2_sched_lock/`, described in `scheduler.md`.
@@ -42,7 +42,6 @@ comparison accepted automatically) and a classic ESP32 devkit runs `esp32central
phase B reads the MITM characteristic with no numeric comparison and no `Deleting unpaired device` line.
- On ESP-IDF 5.5.6 every other reconnect pairs again (`Deleting unpaired device` on the disconnect after a bonded
reconnect), the regression #22081 describes.
#22559 fixes it: the bond survived 15 reconnects and a reset of both boards with no new pairing.
## Bluedroid hex parsing bench (#22081)
@@ -1,47 +0,0 @@
# Bluedroid GATT client command queue
## Queued MTU request recursion overflows BTU_TASK (#17954, #22670)
Bluedroid's `bta_gattc_pop_command_to_send()` runs the head of `p_cmd_list` and removes it only after
`bta_gattc_sm_execute()` returns. When that head is an MTU request (or an execute write) whose send fails at once,
the failure path pops again with the same command still at the head, so it recurses until the BTU task stack
overflows. Every ESP-IDF release so far; PR #22670 fixes it in ESPHome with linker wraps
(`bluedroid_gattc_queue.c`).
Rig, `scripts/bluedroid_mtu_recursion/`:
- Peer: an ESP32-S3 with `btusrv.yaml` (`esp32_ble_server`, one readable and writable characteristic, advertising).
- Central: a classic ESP32 devkit with `btu-common.yaml` plus a per-build file like `btucli-example.yaml` (set
`peer_mac` to the peer's Bluetooth MAC, which is its Wi-Fi MAC + 2, and give each build its own name).
`CONFIG_FREERTOS_WATCHPOINT_END_OF_STACK: y` makes the overflow panic at the write:
`Stack canary watchpoint triggered (BTU_TASK)`.
- Each connection: `on_connect` looks up the characteristic handle right away (a `ble_client` with no child nodes
releases services and cleans the GATT cache in the same SEARCH_CMPL event, so a lookup after a delay finds
nothing), waits 1.5 s, then suspends `BTU_TASK`, calls `esp_ble_gap_disconnect`, queues 4 reads and one MTU
request, waits 30 ms and resumes. The batch is the key: sent one at a time, the BTU task handles each request
before the next arrives and nothing waits in the queue. The reads fail and keep the command slot busy so the MTU
request is queued; when popped, `GATTC_ConfigureMTU` fails with "BT_GATT: connection not established".
`ble_client` reconnects on its own and the next round starts.
- `analyze.py CAPTURE` counts rounds, BTU canary crashes, boots, reconnects with and without a reboot and MTU 517.
- Decode with the IDF `xtensa-esp32-elf-addr2line -pfiaC`; the overflow shows the repeating
`pop_command_to_send (act.c:1739/1746) -> sm_execute -> cfg_mtu (act.c:1042)` frames.
Results on ESP-IDF 5.5.6: dev at 46844d3631 crashed 73 of 73 rounds in 5.5 min (every round, about 4.5 s per
reboot cycle); the PR at ad787163fd ran with no crash and reconnected without a reboot each time. Check the wraps are
linked with `nm firmware.elf | grep __wrap_GATTC_ConfigureMTU` (native ESP-IDF builds only; the define is
`USE_ESP32_BLE_BLUEDROID_GATTC_QUEUE`).
### Execute write variant
`btuxw-example.yaml` sets `exec_write: "true"`, which sends `esp_ble_gattc_execute_write(gattc_if, conn_id, true)`
in place of the MTU request. No prepare write is needed: Bluedroid does not check for prepared writes, and
`GATTC_ExecuteWrite` fails synchronously with the same "connection not established" error. The variant also sets
`CONFIG_BT_LOG_GATT_TRACE_LEVEL_API: y`, so each send logs `BT_GATT: GATTC_ExecuteWrite conn_id=... is_execute=1`,
and those lines are counted per round with awk between the `ROUND n start` lines. In a few rounds (about 3 to 14%)
the controller reports the disconnect before the command is popped. Those rounds log no execute send and never
crash, so count "rounds that reached the send", not all rounds. Most dev backtraces in this variant are cut off in
`vTaskSwitchContext` (the extra trace logging makes task switches more likely at the moment of overflow). Use the
complete ones to decode the `bta_gattc_execute (act.c:1352)` cycle.
Results: dev 46844d3631 had 81 rounds, 70 reached the send, and all 70 crashed. PR e51f0b8321 had 151
rounds, 147 reached the send with exactly one execute line each, and there were 0 crashes and 1 boot.
@@ -26,44 +26,3 @@ swap fails when a round never connects. An owner that never got the link also fa
- Before #22077, case H holds the link for good: 0 adverts, and the address stays in the slot report. With #22077 the
link is released in the same millisecond as the `replaces` warning. In case 4 the first request logs `Connection
request ignored, state: DISCONNECTING` (connected=false at 0.2 s) and the retry connects at 0.5 to 0.7 s.
## Out-of-order GATT discovery responses (#22518)
Some peripherals answer a characteristic discovery (Read By Type) with records out of handle order. ESP-IDF 5.5.6
rejects that (`gatt_process_read_by_type_rsp invalid handle ...`) and ends the whole discovery; 5.5.5, Android and
BlueZ accept it. `scripts/att_reorder/` reproduces it with no such device:
- `server.yaml` (ESP32-S3) serves `FFF0` with six characteristics, then `1826`.
- `proxy.yaml` (classic ESP32, `idf` substitution; keep per-build files such as `example-proxy.yaml` in the same
directory) links `components/attswap`, which wraps Bluedroid's
`l2c_rcv_acl_data` (`-Wl,--wrap`); `attswap.hpp` swaps the first two records of the response whose first record
declares `FFF1`, and logs a swap count.
- `gattget.py PROXY PEER_MAC ROUNDS` reads the services through the proxy like Home Assistant (no cache);
`run.sh PROXY_YAML [ROUNDS]` flashes both boards and prints the services per round and the discovery errors.
On 5.5.6 Home Assistant gets `fff0` with no characteristics and no `1826`, while the proxy still logs
`Service discovery complete` (the proxy ignores the error status); on 5.5.5 every service arrives. To find the order a
real device uses, decode an Android HCI snoop: a Read By Type response lists its records in the order sent.
- Handles, reads and notifies after a reordered response: flash `server-notify.yaml` (`SERVER_YAML` on `run.sh`) and run
`gattcheck.py PROXY PEER_MAC`. It prints every service, characteristic and descriptor handle, reads FFF1, FFF2 and
FFF3 by the received handles (each holds its own value), and enables notify on FFF1, FFF2 and 2ACD. Compare the table
with a run of a proxy built without `attswap`. On ESP-IDF 5.5.5, which accepts the unsorted response, FFF1's
descriptor range runs over FFF2's declaration, value and CCCD, so FFF2 has no CCCD (#22576).
- `bluetooth_gatt_start_notify` only registers the notify on the proxy; the client writes the CCCD itself, as Home
Assistant does. Without that write no notification arrives.
- Give the proxy an `api` encryption key (`API_KEY` for the scripts). A Home Assistant on the same network connected to
a plaintext test proxy and took over its Bluetooth subscription (`Subscription from Home Assistant ... replaces
aioesphomeapi`), dropping the test client's connection mid-run.
## GATT UUID forms through a proxy (#22679)
`scripts/gatt_uuid/`: `server.yaml` (the `ble_notify` server plus a service with 128 bit, 32-bit base and 16 bit
UUIDs on characteristics and descriptors, and 2901 and 2902; set the name with `-s name`), `gattlist.py PROXY PEER
OUT.json` (lists every UUID, reads, notifies 10 s, retries 0x3E), `cmp.py REF.json OTHER.json` (entry by entry diff,
exits 1 on a difference) and `run.sh NAME` (Pico W proxy from `rp2_web_ble/picow.yaml`, list, compare). Take the
reference from an ESP32 proxy (unset `API_KEY` for a plaintext API).
- On rp2 before #22679, BTstack truncated 32-bit base UUIDs with a nonzero low half to 16 bits (`5ec10001-...` came
out as `00000001-...`); a zero low half (5ec10000) came through intact.
- Connects through the Pico W proxy fail with 0x3E now and then (see `reference/boards.md`); the scripts retry.
@@ -1,35 +0,0 @@
# BMP390 / BMP388 (bmp3xx_i2c): register readback, data ready rate, start write faults (#21632, #21633)
Bench: ESP32-S3 devkit (native USB console, ESP-IDF), BMP390 breakout on 3V3, SDA GPIO1, SCL GPIO2, CS and SDO open
(address 0x77), INT to GPIO42. Some breakouts answer chip ID 0x50 (the BMP388 value); the register map is the same.
`scripts/bmp390/`:
- `gen.py NAME [--rate 50Hz] [--iir 128x] [--interval 1s] [--fault --kinds ... --probe-delay MS] [--reset-at 40,80]
[--open-line] [--bus-reset]` writes `NAME.yaml` (no Wi-Fi).
- `bmp390probe.hpp` wraps the sensor's I2C bus (`set_i2c_bus()` from an `on_boot` priority 800 lambda), dumps
CHIP_ID, ERR, INT_CTRL, PWR_CTRL, OSR, ODR and CONFIG before and after setup and every 10 s (`REGS` lines), turns on
data ready on INT after setup (`INT_CTRL = 0x42`, push-pull, active high: the driver never writes INT_CTRL and its
soft reset in setup clears it), counts rising edges (`INT edges=N in M ms`) and logs every pressure value with four
decimals (`READ <ms> <hPa>`).
- `run.sh NAME SECONDS [TAG]` (env `TREE`, `PORT`, `SC`, `PY`) builds, flashes, captures from a fresh boot and runs
`stats.py CAPTURE [SKIP_S]`: read count, cadence, INT rate, detrended noise (21 sample window), repeated values.
Flows:
- Register A/B: the same config on both builds; compare `REGS` after setup and between updates. Forced mode leaves
PWR_CTRL at 0x03 (sleep, both sensors enabled), normal mode reads 0x33. ODR codes: 0x00 200 Hz, 0x02 50 Hz, 0x07
1.5 Hz, 0x0B 0.1 Hz (period 5 ms << code).
- Real output rate: INT edges per 10 s window. Forced mode gives one edge per update.
- Settling: the bench cannot make a pressure step, but the soft reset in setup restarts the IIR filter with an
offset, so the first readings after boot show its step response (50 Hz with 128x settled in about 8 s).
- Reset behind the driver: `--reset-at` writes CMD 0x7E = 0xB6 at those seconds; a driver that handles it logs that
the sensor was reset and restores the registers.
- Start write faults (`--fault --kinds`, one letter per event at 30 s and then every 20 s): `N` the next start write
NACKs in software, `M` every start write NACKs for 5 s, `T` SDA held low through the GPIO matrix for the next start
write, `W` the same for every transfer for 5 s, `L` a continuous 5 s hold, `S` a 300 ms hold, `H` and `D` held
transfers of the harness only. `PROBE` lines check the bus `--probe-delay` ms later.
Pitfall: on the S3 with ESP-IDF 5.5.6, any transfer that fails while SDA is routed to a GPIO held low (or routed away
with the line floating high) leaves the `i2c_master` driver hung on every build, with or without the sensor: every
later transfer logs `I2C software timeout` while the lines read idle high and the saved pad registers are back, and
`i2c_master_bus_reset()` does not help. Use the software NACK kinds (`N`, `M`) to test what a component does after a
failure and the recovery; use the wire kinds only to show the first failure.
@@ -1,17 +0,0 @@
# cc1101
## Sleep and wake checks read from the chip (#22519)
`scripts/cc1101_sleep/`: `cc1101-sleep.yaml` runs an ESP32-S3 (ESP-IDF) with an E07-M1101D module on SPI and runs
tests T0 to T6 4 s after boot; `cctest.hpp` reads every config register and the PATABLE directly over SPI. The log
ends with `DONE fails=N`, and T6 prints whether each setter woke a sleeping chip.
- Proof that the chip sleeps: with CSn low, SO (MISO) stays high until the crystal runs, so it is high after
`power_down()` and low at once when awake. PATABLE entries 1 to 7 are also lost in SLEEP, so write marks to them
before `power_down()` and check that they are gone.
- Selecting the chip to check it wakes it, so after the check the chip is awake while the component still thinks it
sleeps. Call `wake_up()` and `begin_rx()` before the next case.
- Leave FSCAL3 to FSCAL1 (0x23 to 0x25), FSTEST (0x29) and AGCTEST (0x2B) out of register comparisons: calibration
and the chip itself change them.
- Results: on 7e6b90d08b, `set_frequency`, `set_channel` and `set_modulation_type` woke a sleeping chip and left it
in RX. 9e65d0c327 fixed that; every other check passed on both, and 200 sleep and wake cycles took about 1 s.
@@ -1,7 +0,0 @@
# combination
## NaN from the last source (#22510)
A combination sensor defers each update and uses the value of the source that published last. A NaN from that
source drops the whole update, so when testing NaN handling keep the last-publishing source finite, and compare
rows one update late. Heap churn per update: see `heap_alloc.md`.
@@ -24,11 +24,3 @@ put `scripts/` on `PYTHONPATH`):
- Quick crash reboots count toward BK72xx safe mode, so space out back-to-back crash tests on those boards.
- A device that keeps the previous crash replays it to every new log client until the next crash; that is one
record, not two.
## Crash records and log subscribers
The record is replayed and cleared by the first log subscribe (`on_subscribe_logs_request`), so drain it on the
first boot after an OTA, or a stale record from the previous build shows up. Moving `api: port:` keeps Home Assistant
off the board while testing. `scripts/crash_report/quiet_then_subscribe.py HOST PORT OUT N` drains the record, stays
connected without a log subscription for N reboots, then subscribes and records the replayed record; it tells a crash
that needs a log subscriber (like the W5500 IPv6 reboot abort in `reference/boards.md`) from one that does not.
@@ -1,13 +0,0 @@
# crc32 helper and the signed OTA block check (esphome#22398, 2026-11-07)
`scripts/crc32/crctest.hpp` checks `esphome::crc32()` against Python `zlib.crc32` values: `"123456789"` = `0xCBF43926`,
empty input returns the seed, chained calls (4 + 5, 300 one-byte calls, 97-byte chunks from an odd offset), bytes
0..255, 32 × 0xFF, 4093 bytes from offsets 0 to 3, and 8 KiB timed with `(esphome::micros)()` (best of five).
`common-crc.yaml` runs it at boot and every 20 s and also makes an unqualified `crc32(...)` call from a lambda, which
catches a clash with anything the generated `main.cpp` pulls in. Paths and timings seen (8 KiB): ESP32 ROM 512 µs,
ESP8266 PROGMEM nybble table 3072 µs, RP2350 932 µs, BW15 (FlashDB) 656 µs, BW12 2000 µs, BK7238 1026 µs.
The signed OTA signature block check (`block_is_valid()`), the PR's only converted caller, ran the accept and reject
cases in `ota.md` (signed OTA with several trusted keys).
Results: https://github.com/esphome/esphome/pull/22398#issuecomment-6716330334 and #issuecomment-6716545831.
@@ -13,6 +13,8 @@ replies.
- `run.sh LABEL` builds, flashes, captures 90 s and writes `out-ezo<LABEL>.events`. Compare dev and the PR: the
replies, callback texts, read values and the wait before each read (912 ms for reads and calibration, 304 ms for
the rest) must match; differences in queue limits are the PR's intended change.
- Call plain `millis()` in a fake: Arduino builds define a `millis()` macro (`esphome::millis()`), so
`esphome::millis()` breaks there, and on ESP-IDF the plain name resolves through `using namespace esphome`.
- Found with it: dev read past its 32-byte buffer on an unterminated reply (a custom callback got 56 bytes), and
two queued address changes both went to the first new address.
- Results: https://github.com/esphome/esphome/pull/21840#issuecomment-6493565259
@@ -1,41 +0,0 @@
# Heap allocation counts and callers (#22509, #22510, ratgdo/esphome-ratgdo#705)
## Counting
Replace the global `operator new` in an `esphome: includes:` header on ESP32 (Arduino or ESP-IDF) and log the
delta from an `interval`. Compare a PR build with its base on the same flow; the rate per second shows churn.
- The ESP8266 core defines `operator new` non-weak. `-Wl,--wrap=_Znwj` worked in a sensor-only config but
boot-looped a neopixelbus config (alignment fault in the wrapper), so report free heap only there.
- `vec = {}` keeps a std::vector's capacity (it is an initializer_list assignment); only
`std::vector<T>().swap(vec)` or `clear()` plus `shrink_to_fit()` frees it. Seen as heap that did not come back
after a WLED effect stopped.
## Callers
`scripts/heap_alloc/`:
- `allocring.hpp` counts every `operator new` and, once armed, keeps the first 1024 with their size, time and
five caller PCs, about 28 KB of RAM. It disarms itself when full. The frame walker
(`esp_backtrace_get_next_frame`) exists only on Xtensa (ESP32, S2, S3); on RISC-V chips entries keep size and
time without callers. Aligned `operator new` (`std::align_val_t`) is not replaced, so it is not counted;
nothrow `new` goes through the replaced one.
- `allocring.yaml` (package, substitution `alloc_arm_s`) arms it that many seconds after boot and adds a
"Dump allocations" button that logs what was kept. The press stops recording until the next reboot, so
reboot (or reflash) before each run; a second press returns the same entries.
- `alloc_marker(a, b)` from a lambda writes a marker entry, for example on each cover operation change, so the
dump splits into phases.
- `dump.py HOST OUT` presses the button and writes the dump to a fresh OUT, pressing again until the header
and every entry arrived (a late log subscription or dropped log lines). `decode.py LOG ELF [ADDR2LINE]` prints each entry with its callers as function and file:line, and exits 2
when the recorder never armed, an entry was still being written, entries are missing, repeated or out of
range, no Xtensa entry has a caller, the ELF is from another build (the buffer address in the header differs), or the buffer
filled. A PR build and its base can share that address, so copy each build's `firmware.elf` next to its dump. On RISC-V it prints sizes and times only. It reads only the newest dump in the file. The ESP-IDF ELF is `.esphome/build/<name>/build/firmware.elf`.
- `cycles_cover.py HOST N OUT` waits for the "Allocation recorder armed" binary sensor, then moves the first
cover to the far end and back N times (`cover_command(position=...)`, waiting for idle each way) with a states-only API client, then saves the dump the same way. It exits 2 when a move
never reached its position.
Findings:
- Arm well after boot: the scheduler regrows its storage once after the freelist trim (see `scheduler.md`).
- Repeat the action several times in one session and count only what comes back. On a ratgdo cover (an external
component, ESP32 ESP-IDF) the first door open after boot allocated 18 times: four 36 B SchedulerItems,
`to_add_` and `items_` growth, and the API `DeferredBatch` vector. Cycles 2 and 3 allocated nothing.
- Keep the API client out of the count (see `api.md`): connect before the run, subscribe to states only, and
subscribe to logs only for the dump.
@@ -45,44 +45,3 @@ without the fix logs `Applying the report that arrived during the transition` an
ends at 100% on both builds because the last call's target differs from the held report, so it is a regression
check only, not a reproduction. Separate button presses are too slow (about 1.5 s each) to land inside the fade.
The mirrors are not API entities, so read their state from the "Log state" button's line. Results: https://github.com/esphome/esphome/pull/21144#issuecomment-6278257695
## Wizard-linked homeassistant entities on every platform (#21908)
`scripts/homeassistant_wizard/`: `ha-all-common.yaml` adds to the buttons of `ha-button-common.yaml` one entity with a
fixed `entity_id` and one linked to an `api: wizard:` input for each of sensor (plus one with `attribute:`),
binary_sensor, text_sensor, number, switch, select, text and light (the wizard light is in `ha-wiz-light.yaml`), and
logs the ESP8266 heap every 5 s (`ramchk`). `hadrive.py HOST` plays Home Assistant over a plaintext API connection: it
records the state subscriptions the device asks for, sends states (including number and text min, max, step and mode,
select options and light supported_color_modes), commands every writer, sets each wizard input with
`WizardInputSetRequest` (key = FNV-1 hash of the input id), changes it to a second ID in another domain, sends an
invalid one and reconnects. It prints SUB, STATE and ACTION lines that diff cleanly between two builds, and PASS or
FAIL. `run.sh LABEL` (env `TREE`, `PORT`, `SC`, `PY`; `NO_LIGHT=1` for a build without #21908) flashes a D1 mini, takes
three idle boots of heap readings and drives one boot. `lightcheck.py HOST` checks again that the wizard light follows
only its current ID.
- Builds before #21908 cannot compile a wizard-linked homeassistant light (the wizard called `set_entity_id()` on the
LightState); `NO_LIGHT=1` leaves `ha-wiz-light.yaml` out.
- An unset wizard number or select rejects a command in the core (`> max 0`, `has no options`) before the
`No entity ID set yet` check; nothing is sent either way.
- Field 3 of LightStateResponse is brightness, not `missing_state`; SwitchStateResponse has no missing flag.
- A light publishes only changes, so a report of the state it already has publishes nothing.
## homeassistant switch on a script, host build against a throwaway Home Assistant (#22548)
No board is needed when the change is in validation or platform independent code: an ESPHome `host:` build serves the
API on 127.0.0.1:6053. `scripts/homeassistant_script_switch/`:
- `ha-configuration.yaml` is the Home Assistant config (port 8129, a 20 s `long_job` script that sets and clears
`input_boolean.script_ran`, a `quick_job` toggle script). Onboard it with `scripts/mqtt/ha_discovery/onboard.py DIR`.
- `hsw.yaml` mirrors both scripts and the flag with homeassistant switches, with template buttons that call
`switch.turn_on` and `switch.turn_off`.
- `haflow.py TOKENFILE` adds the device with the esphome config flow and turns on `allow_service_calls` in the options
flow; without it the device's action calls are refused.
- `drive.py TOKENFILE NAME` presses the buttons through the REST API and prints the script and flag states after each
step.
Notes:
- homeassistant import entities default to `internal: true`, so the mirrors are not Home Assistant entities; read
them from the device's `Got state` log lines.
- Turning a script off stops it mid run (the flag stays on). A script that finishes at once reports on, then off, in
the same second.
@@ -1,20 +0,0 @@
# I2C master interrupt storm and stuck bus faults (esphome#22270 / #22396, 2026-11-06 to 11-07)
`scripts/i2c_isr_fault/inasim.hpp` is a software INA219 at 0x40 answered on the ESP32 master's own pads (SDA GPIO32,
SCL GPIO33, nothing wired): the slave pulls a line low by switching the pad's output mux from the I2C0 signal to plain
GPIO output 0 (a wired AND with the master), from a GPIO ISR on core 0 that spins through one transaction. A core 0
task re-asserts long stuck lines every tick. From 20 s after boot it injects seeded faults into `p_fault` per mille of
transactions (garbage bits, NACK, clock stretch, SCL/SDA held low up to `hold_max_ms`, glitches, vanish).
`i2cbase.yaml` (package): `i2c:` at `freq` with `scan: true`, an `ina219` polled every `ina_interval`, a bus voltage
register read every `hammer`, a boot self test, and 30 s `MASTER`/`SLAVE` counter lines (ok, timeouts by kind,
garbage, longest failed run, loop stalls). `example.yaml` shows the substitutions: framework and version (`fw`,
`fwver`), `seed` (seed 1 at 5 % hangs dev with an interrupt watchdog on CPU0 at about 245 s, the #22270 report),
`p_fault`, `fault_mask`. `run.sh LABEL [SECONDS]` builds, flashes and captures (640 s default); `summary.sh` prints
crashes, watchdogs, timeout kinds and the minimum ok reads per window.
Findings: ESP-IDF 5.4.2 has a different failure (the driver waits for bus idle with no timeout after a NACK, so the task
watchdog reboots in a loop with or without the guard); 5.5.5, 5.5.6 and 6.0.1 hang on the storm without the guard and
run clean with it. ESPHome never uses the driver's async mode, so there is no asynchronous path to test.
Results: https://github.com/esphome/esphome/pull/22396#issuecomment-6713396173
@@ -1,66 +0,0 @@
# Improv serial and BLE (#22597)
`scripts/improv/` drives Improv on a freshly erased board with no saved network:
- `improvcli.py PORT OUT --secrets FILE`: Improv serial client. It shares the console with the logs and parses
frames out of the log stream. The flow: 300 stray bytes with no `I` in them, state, info, network state, scan, a
bad password (expect `unable_to_connect` after about 30 s), the real password, then state and network state
again. It ends with a count of frames, bad frames, port resets, reboots and log lines. Credentials come from the
secrets file and are never printed, and the SSID is masked in scan results. It reopens a native USB port that
re-enumerates. A reboot is counted from each new `Running through setup()` log line after the settle, so a board
behind a USB-UART bridge that reboots without a port reset is caught too.
Every request must get its reply; a missing reply, an unexpected error, a malformed frame, a port reset or a
reboot fails the run. `--skip-bad` leaves out the bad password case, `--keep-dtr` leaves DTR asserted,
`--allow-reset` accepts a reboot.
- `improvble.py NAME SECRETS`: Improv BLE client (bleak). Finds the device by name, sends WIFI_SETTINGS and prints
the state changes and the RPC result URLs. Needs `improv_ble` (formerly `esp32_improv`) with `authorizer: none`
and a `next_url`; the result must carry a URL.
- `common.yaml`: package with Wi-Fi and no network, `web_server`, `improv_serial` with a `next_url` (so the
provisioned reply carries URLs) and an ALIVE line every 5 s. `example-esp32.yaml` uses it on a classic ESP32.
- `run.sh NAME [improvcli args]` (env `TREE`, `PORT`, `SECRETS`, `PY`): compile, erase, flash and run the client.
It erases with esptool, so ESP chips only; on LibreTiny or RP2040 erase and flash with that platform's tools and
run `improvcli.py` directly. Both clients end with PASS or FAIL and exit non-zero on FAIL.
Per board:
- Improv serial uses the logger's UART, so the client must open the port the log is on. On LibreTiny check which
UART `logger: hardware_uart: DEFAULT` resolves to (see the board notes).
- BW15: drop `web_server:` or add an `ap:` (it resets during setup otherwise), and pass `--allow-reset` for the known
reboot after `Disabling AP`; the provisioned reply usually goes out first, not always. See `reference/boards.md`.
- ESP32-C3 on USB_SERIAL_JTAG: passes with and without `--keep-dtr`; its reboots only show in the
`Running through setup()` count (see `reference/boards.md`). The scan returns only the last scan's results, which
are empty in the first seconds after boot; use `--settle 10` or more when the target network should be listed.
- Pico 2 W: run with `--keep-dtr` (see `reference/boards.md`). Its Improv scan lists no networks, on dev and PR alike.
- Erase the flash between builds: a saved network skips the provisioning path entirely. Both clients fail at once
when the board already reports `provisioned`; erase the flash first. A new config name does not help: the
network is saved under a fixed key when the config has no `networks:`. A provisioned board also stops advertising
Improv over BLE, so `improvble.py` reports the device as not found.
- If the real password fails right after the bad password case, rerun once before blaming the change: on the C3 it
failed once (`Auth Leave` while Wi-Fi was still retrying the bad network) and passed on every rerun.
When posting results:
- Scan results list the neighbours' SSIDs; filter `RPC cmd=4 len=[1-9]` lines out of anything you post.
- Mask IP addresses in the result URLs.
- Cover a long password too: provision once against a network whose password is 20 characters or more (the client
prints the length).
## Improv serial scan reply over the logger UART (#22758)
- `scripts/improv/improvscan.py PORT OUT [--settle S]` (on `improvcli.py`'s frame parser) sends an Improv RPC request
scan (command 0x04) and records each reply frame (checksum, strings, auth field bytes and length) until the
end-of-list reply. Opening the port resets a D1 mini and a classic ESP32 (see `reference/boards.md`), so it waits
30 s before sending.
- A connected device frees its scan list, so the reply is just the end-of-list frame. Add a `wifi_info:`
`scan_results:` text sensor to keep the list.
- For an auth `NO` entry you need an open network in range: run an AP-only build without a password on a second board
(`scripts/improv/openap32.yaml`).
- Scan replies list the neighbours' SSIDs; filter them out of anything you post.
## Log macro rewrites, A/B by log text (#22758)
`scripts/log_macros/`: `lm-common.yaml` (one entity of every common platform plus `improv_serial` and `web_server`),
`drive.py HOST OUT` (presses the exercise button, reads API sensor states and web_server JSON) and `norm.py SERIAL`
(keeps config dump lines, `lm` lines and state lines, and replaces addresses, the device name, signal, setup times
and per-boot values with placeholders instead of dropping the lines) so two builds' captures diff cleanly. Recent ESPHome logs sensor states and switch, light and fan calls only at VERBOSE; at
DEBUG a log diff sees none of them, so `lm-common.yaml` logs at VERBOSE.
@@ -1,28 +0,0 @@
# IPv6 link-local and SLAAC on ESP32 Wi-Fi: boot timing and race injection (#22364)
`scripts/ipv6_boot/` (ESP32-S3 and ESP32-C3, native USB console, ESP-IDF). One YAML per build
(`example-s3.yaml`, `example-c3.yaml` show the shape) with the substitutions `min_count` (network
`min_ipv6_addr_count`) and `cap_ms`.
- `v6probe.hpp` and `v6common.yaml`: `network: enable_ipv6`, VERBOSE logs, and a 50 ms interval that logs milestones
from boot (`MS assoc|ipv4|linklocal|slaac|connected <ms>`; addresses come from `esp_netif_get_all_ip6`, so they
count once they are preferred, after duplicate address detection), then `RESULT boot ...` with `heap_used`, then
disables Wi-Fi for 3 s and logs `RESULT reconnect ...` the same way, then reboots. A phase ends when SLAAC and the
connection are both seen, or after `cap_ms` (`timeout=1`).
- `v6hf.yaml` adds a `HighFrequencyLoopRequester` (a busy main loop, as with a display or audio).
- `v6inject.yaml` links with `-Wl,--wrap=esp_netif_action_connected` and defines `V6_INJECT`; the wrapper in
`v6probe.hpp` sleeps 300 ms in the event task before esp-netif brings the station interface up, after the wifi
component already queued STA_CONNECTED. That makes the "interface not up yet" race happen on every connect.
- `run.sh NAME SECONDS` (env `TREE`, `PORT`, `SC`, `PY`) builds `NAME.yaml` as device `NAME`, flashes, captures the
self-rebooting boots and runs `an.py CAPTURE...`: per phase the `esp_netif_create_ip6_linklocal failed` count,
boots without link-local, SLAAC or a connection, connection timeouts, and median and worst times.
- Check the network first: a computer on the same network shows an `autoconf` IPv6 address when a router advertises
a prefix. SLAAC arrived about 2 s after link-local on the test network.
- The natural race did not show: no failure in about 160 boots on the S3 (normal and busy loop) and the C3 without
the fix. esp_event runs handlers in registration order and the wifi component registers before esp-netif, but the
event task reaches esp-netif's handler within microseconds. Only the injected delay reproduced the failure.
- An extra event handler registered from `on_boot` cannot inject the race: it runs after esp-netif's handler, when
the interface is already up. Wrap the esp-netif call instead.
- `wifi.disable` removes the addresses, so the reconnect phase creates link-local again on the same path.
- Count failures by the probe's `ll=0` too, not only by the warning: the USB console drops lines.
@@ -61,10 +61,3 @@ frame of two LEDs set to `Color(0xA5, 0x3C, 0x0F)` reads `3ca50f3ca50f`.
- An RP2350 `panic()` escalates to a HardFault and the rp2 crash handler reboots within about a second, so a panic
during component setup boot-loops before USB CDC enumerates and the board looks dead (no port, no BOOTSEL drive).
Hold BOOTSEL while plugging it in and flash the UF2 drive. To see such a fault, trigger it after boot.
## Counting a PIO strip's edges on its own pin (#16243)
A GPIO interrupt on the strip's own data pin (`gpio_set_irq_enabled_with_callback(pin, GPIO_IRQ_EDGE_RISE, ...)`)
counts every bit the PIO program sends, with no wires or LEDs; a strip that sends nothing shows 0 edges per report.
Log the PIO block's state machine enable mask and program counters next to the count. `pio_probe/pio-common.yaml`
clears PIO1 instruction memory at boot because it survives a reflash (see `reference/boards.md`).
@@ -160,21 +160,3 @@ auth frames.
Known LibreTiny behaviour seen here: after a join with a BSSID and channel, LibreTiny's partial-scan reset leaves
the next SSID-only join with nothing to scan (`RTW_NONE_NETWORK` in about 18 ms). That is a firmware bug, not
the board; passing channel 0 avoids it.
## Pulse counting and edge interrupts on LibreTiny (esphome#22391, #22400, #22403, #22404, 2026-11-06 to 11-07)
`scripts/libretiny_edges/`:
- `pulse-bk.yaml` + `pulsegen.hpp` (BK7238, generator pins `GEN_PIN_A`/`GEN_PIN_B` jumpered to the counted pins) and
`pulse-rtl.yaml` + the same header (BW15, mbed `pwmout` on `GEN_PIN_NAME`): one pulse shape per 35 s stage, from
BL0937-like 38 µs pulses at 1 kHz up to 50 ms, counted by `pulse_counter` with `inverted` and filters as
substitutions; `meas` lines give count per stage. Found: Beken counted half of BL0937 pulses (readings halve, so a
calibration note was needed), RTL8720C counted none, RTL8710B none with inverted pins, LN882H counted correctly.
- `bw12-base.yaml` with two pin layouts (`common-bw12.yaml`, `common-bw12-b.yaml`) + `bwdbg.hpp` (BW12, RTL8710B): edges
without wiring by toggling the pad pull on floating pins (`PAD_PullCtrl`), `-Wl,--wrap=digitalRead` to count ISR
versus task reads, and a pin/IRQ register dump; `summ.py` summarizes. A jumper IO12 to IO13 on the jig gave a real
driven edge.
- `common-bw12-time.yaml`: `millis()`/`micros()` every 5 s against host time, a DWT cycle bench per call (also under
`InterruptLock`), and a mask ROM dump of `SYSTIMER_TickGet`.
Results: #22400 https://github.com/esphome/esphome/pull/22400#issuecomment-6712520146, #22403
https://github.com/esphome/esphome/pull/22403#issuecomment-6712901242, #22404
https://github.com/esphome/esphome/pull/22404#issuecomment-6713095035.
@@ -34,11 +34,3 @@ wave on the same pad no longer reaches the input (the pad reads 0).
- Result on #21074 at 2ef417bd81 with peripherals powered down: the PR caught all 644 real edges (dev 66) but added
4642 calls while the line was high, one per sleep cycle (about 16 ms), with the level read correctly. Peripherals
on: both builds caught every edge. https://github.com/esphome/esphome/pull/21074#issuecomment-6503939850
## Power lock check
`power_management: esp32_power_management` with `profiling: true` and an `interval` lambda that calls
`esp_pm_dump_locks(stdout);` prints the lock and mode tables on the USB console (not through the ESPHome logger).
Compare lock lists between builds: `usb_serial_jtag` holds `NO_LIGHT_SLEEP` by itself. For a soak that the USB console
must not keep awake, set `logger: hardware_uart: UART0`, dump the locks into `open_memstream()` and log the SLEEP,
CPU_MAX and light sleep count lines, read over the API.
@@ -1,17 +0,0 @@
# MCP23017 IOCON and interrupt line (#22232)
Bench: an ESP8266 (Feather HUZZAH) with an MCP23017 breakout at 0x27, SDA GPIO4, SCL GPIO5, INTA to GPIO14, INTB to
GPIO13, PA0 jumpered to PB0 and PA1 to PB1, no extra pull-ups (the breakout's were enough at 100 kHz). The esptool
reset keeps the expander powered, so a reflash shows whether register bits from the previous build survive.
`scripts/mcp23017/`: the `mcptest` external component (`ext/mcptest`) reads IOCON at 0x0A and 0x0B 6 s after boot,
toggles PA0 and PA1 20 times, samples the INT levels right after and 200 ms after each write, counts INTB edges in
an interrupt, checks the PB binary sensors (`SUMMARY` line), then checks whether IOCONB aliases IOCONA
(`IOCONB_TEST`). `int.yaml` wires INTA to the hub's `interrupt_pin`, open drain or push-pull by the `open_drain`
substitution (variants `int-od` and `int-pp`); `no-int.yaml` has no interrupt pin and counts INTA edges in
`mcptest`. `run.sh VARIANT LABEL` (env `TREE`, `PORT`, `SC`, `PY`) builds, flashes and captures 14 s.
- In BANK=0, 0x0A and 0x0B are the same register: a write to one reads back from the other.
- A push-pull INT line works without a pull-up. Open drain with no pull-up stays low after the first interrupt
(ESP8266 INPUT), so the hub loop never stops and the inputs still update by polling.
- Builds before #22232 leave ODR and MIRROR set across reflashes.
@@ -1,33 +0,0 @@
# MCP2518FD CAN FD over SPI (esphome#21951, 2026-11-07)
Bench: ESP32-PICO-D4 (`board: pico32`, ESP-IDF) wired to an MCP2518FD module with an ATA6563 transceiver: SCK GPIO25,
SDI GPIO26, SDO GPIO27, nCS GPIO32, INT GPIO33, module 3V3/GND, CLK pin to GPIO34. No second CAN node, so everything
runs in `LOOPBACK`, or with the controller forced into internal loopback behind the driver.
- **Read the crystal.** The module's can is marked `H20.00` (20 MHz) even though listings say "8 Mbps"; the PR default
is `40MHZ`. Loopback never leaves the chip, so a wrong `clock:` still passes at half the configured rates. Measure it:
`pulse_counter` on the CLK pin with `internal_filter: 0us` reads SYSCLK/10 (2 MHz) at reset. Older heads wrote 0 to
`OSC` byte 0, which set `CLKODIV` to /1 (CLKO = 20 MHz, still countable by PCNT).
- `scripts/mcp251xfd/canlb.hpp`: stream checker (one frame in flight, burst of 8 every 64th; standard/extended IDs, FD
with and without BRS, every 4th classic, DLC 0 to 15, sequence in the payload; starts 5 s after boot so the
`dump_config` stall cannot trip the 50 ms in-flight timeout), a queue checker for bursts, RTR and TX objects, raw SPI
register access on the driver's own `SPIDevice` (`rd32`, `wr8`, `chip_reset` = `0x00 0x00`), forced modes
(`force_mode` writes `CON+3`; `force_via_config` goes through Configuration so the chip accepts it), `watch_start`
(polls `CON` every 2 ms until `OPMOD` and `REQOP` equal the target, logs the time), `txcheck` (reads each TX object
back from chip RAM after a frame full of 0xAA, padding included), `burst` (sends N frames in one call so the loop
cannot drain RX: FD depth 20, CLASSIC depth 32 + one overrun) and a PCNT unit on the CS pin, which counts SPI
transactions while the pin stays an output (`cs_mark`/`cs_report`: idle cost per second).
- `common-can.yaml` is the package; each config adds its `canbus` block and a `phases` script: `can-loopback.yaml`
(stream, RTR, TX objects, idle cost, chip reset, two refused `REQOP` writes), `can-classic.yaml` (CLASSIC with a
forced internal-loopback window: force once to trigger the driver's recovery, force again right after it restores,
then burst 48 and RTR inside the next 5 s check interval), `can-normal-busoff.yaml` (no node acknowledges, so the
controller cycles error passive/bus-off; each forced fault and reset waits for TREC `TXBO`).
`run.sh CONFIG LABEL [SECONDS]` builds, flashes, captures and prints the summary lines.
- Gotchas found: a refused `REQOP` (LISTENONLY or Restricted from LOOPBACK) leaves `OPMOD` unchanged and stalls TX;
the chip refuses LOOPBACK to CLASSIC directly (go through Configuration); after a chip reset TREC reads `0x00200000`
(`TXBO` set in Configuration mode), so a driver that skips recovery on bus-off never recovers a reset during an error
state. Neither a reset nor a refused request sets an interrupt flag, so detection waits for the periodic check.
Results: https://github.com/esphome/esphome/pull/21951#issuecomment-6715452893 (first run),
#issuecomment-6716105991 (TX objects, idle cost), #issuecomment-6716741859 (reset during bus-off bug),
#issuecomment-6716957543 (fixed).
@@ -16,64 +16,3 @@ states, saves the device log). Before 8cc8df4a1f a modem only config with `api:`
config dependent `AUTO_LOAD` added `network` after the dependency check; an explicit `network:` block was the
workaround. Results: https://github.com/esphome/esphome/pull/6721#issuecomment-6347395159 and
https://github.com/esphome/esphome/pull/6721#issuecomment-6348520416
## SIM7080G on an M5Stack Atom Lite (#6721)
The M5 Unit CatM (SIM7080G) on the Atom Lite's Grove port: RX GPIO32, TX GPIO26, `model: SIM7080`.
`templates/modem/modem-passthrough.yaml` is a raw UART passthrough with both directions logged, and
`scripts/modem/mdrive.py HOST LOGFILE text:AT+CPIN? wait:5 ...` sends AT commands through it and saves the
replies, to check what the module does with no component in the way.
- With no SIM, the component sets the `SIM not ready` warning and polls `AT+CPIN?` once a second.
- With a SIM that has no data plan, `AT+CGATT=1` blocks the AT channel for about 141 s and then answers `ERROR`.
The module answers nothing meanwhile, so the component logs `Network attachment request refused` and then `Modem not
responding`, and loops every 2 minutes 22 seconds.
- `AT+CFUN=1,1` leaves the module silent for about 2.5 minutes. It comes back without a power cycle.
- A module left in CMUX (`AT+CMUX=0`, then an ESP restart) recovered 10 of 10 times (`Autodetect mode: 5`).
- PPP and a warm reboot during PPP need a SIM with a working data plan.
### SIM7080G with a live SIM
- A SIM whose plan does not cover LTE-M or NB-IoT never registers on the SIM7080G: CEREG goes 0 (not searching) right
after CFUN=1, 2 (searching) for about 60 s, then 3 (denied) for a minute or two, then searching again, with CSQ 19 to
31. COPS=? lists the Cat-M networks; manual COPS fails (ERROR or denied), so don't retry it. PPP, attach timing and a
warm reboot during PPP need a SIM whose plan covers LTE-M.
- esp_modem 2.1.0 sends `AT+CGATT=1`, `AT+CGATT?` and `AT+CEREG?` with a 500 ms timeout. Unregistered, `CGATT=1`
holds the AT channel (15.6 s denied in the passthrough, about 40 s in the component, over 2 minutes with a SIM that
has no data plan) and the module drops every command sent meanwhile: no echo, no reply (an `AT+CMUX=0` sent then is
lost).
- `ATD*99#` sometimes answers `CONNECT 150000000` without an attach. The module then sits in data mode, sends no LCP,
ignores AT for minutes, and esp_modem's autodetect returns 0 forever (dev logs the wiring hint). A guarded `+++`
(1.1 s silence, `+++`, 1.1 s) returns OK at once; give the passthrough an `Escape` template button doing
`write_str("+++")` between two 1200 ms delays.
- `AT+CFUN=0` powers the SIM down only about 1.7 s after OK (`+CPIN: NOT READY` URC), and the component reads
`+CPIN: READY` before that, so `init_at: [AT+CFUN=0]` cannot fake a missing SIM. There is no reliable no-SIM path
without removing the SIM.
- `AT+CFUN=1,1` left the module silent for 20 s to 2.5 min.
- Pins that reach nothing, for the no-modem path on an M5Stack Atom (or any ESP32): `rx_pin: GPIO22`, `tx_pin: GPIO19`.
- Drive the component build with send_at buttons (`AT DIAL` = `ATD*99#`, `AT CFUN1`, `AT+CMGR=9` for an error text)
and the `Restart` button; `mdrive.py press:NAME`. A `send_at` needs about 3 s before a restart to go out.
## SIM7670G on the Waveshare ESP32-S3-SIM7670G-4G (#22581)
- Board: ESP32-S3 console on a CH343 (UART0, so `logger: hardware_uart: UART0`; the default USB logger prints
nothing). Modem UART: ESP TX GPIO18, ESP RX GPIO17, 115200, `model: SIM7670`. The modem is powered by the "4G" DIP
switch, no GPIO power-up needed. Module SIM7670G-MNGV, firmware V1.9.05.
- The module's own USB enumerates as four `05c6:9330` ports; one of them is an AT port that works next to the
component's UART (another is a binary diag stream). Send AT lines there to read CEREG, CGDCONT, CEER or SMS while the
ESP runs. Run AT experiments with an idle ESP image (a hex passthrough), or the component's own reconnects (CFUN=4,
CGDCONT) get mixed into the results.
- Hex passthrough with frame buttons: `uart: debug: after: timeout: 40ms` with `log_hex`, buttons that `uart.write`
SABM/DISC/CLD/AT frames per DLCI (FCS: CRC-8 reversed poly 0xE0 over address, control, length, then 0xFF minus it)
and buttons that write runs of 0xF9.
- Findings: echo is on after power-up or an AT channel reset, and esp_modem's mode guess takes the echoed LCP probe for
a PPP reply (`Autodetect mode: 2`), the start of #22581. After an ESP reset during PPP the module stays in CMUX and
ignores SABM/DISC/CLD/AT/+++ (it answers SABM0 with a UA only now and then); a burst of 4 KB of 0xF9 usually makes it
print `QCRDY` and fall back to command mode with echo on (worked 10 times, failed 4 tries once after a reflash).
- A PDP context written at CFUN=4 makes the next attach fail with `+CEER: EMM_CAUSE_ESM_FAILURE` (CEREG 3) until a
CFUN=0 cycle; written at CFUN=0 it attaches in about 8 s. After each attach the network can rewrite context 1's APN to
its own name. An empty `apn: ""` validates and attaches (the network picks the APN).
- A SIM can attach and get an IP and DNS while no TCP or ICMP traffic comes back, through PPP (CMUX and data mode) and
through the module's own `AT+HTTPACTION` (706) and `AT+CPING`: that is the carrier side, not the component.
- SMS: when the SIM's SM store is full, set the receive store to ME with `AT+CPMS="SM","SM","ME"`. Messages are UCS2
hex in text mode; decode in memory only, never into a log, and keep SIM data (numbers, identifiers) out of results.
@@ -1,33 +0,0 @@
# network.priority DNS loss (#19805, espressif/esp-idf#19082)
With two interfaces in `network: priority:` on ESP32, ESPHome sets CONFIG_ESP_NETIF_SET_DNS_PER_DEFAULT_NETIF and pins
the default with `esp_netif_set_default_netif()`. In that mode a DHCP ACK only fills the netif's own DNS cache; lwIP's
global table is written only by `esp_netif_set_default_netif_internal()`. `esp_netif_dhcpc_start()` and
`esp_netif_set_ip_info()` (DHCP client stopped) on ANY netif call `dns_clear_servers(true)` on the global table, and the
pin makes IDF skip re-election, so the table stays empty until the app re-asserts the default. Lookups then fail at once
with getaddrinfo 202 (lwIP ERR_VAL, no server set). Unfixed on IDF master when this was written.
Triggers: Wi-Fi joining or reconnecting while Ethernet is the default (ESPHome's `wifi_sta_ip_config_` and IDF's
`esp_netif_action_connected` start DHCP); Ethernet with no link while Wi-Fi is the default (the 15 s
`Connecting failed; reconnecting` retry runs `set_ip_info` plus `dhcpc_start` every time).
Harness: `templates/network_dns/dnsrepro.yaml` + `dnsdiag.hpp` for the Waveshare ESP32-S3-ETH (W5500). Every 3 s a `DIAG`
line classifies the global DNS slots against each netif's cache (0 empty, E eth's, W wifi's, B both, X other) and names
the IDF default and lwIP `netif_default`; every 5 s a `LOOKUP` resolves a fresh `*.10.0.0.1.nip.io` name (a public
wildcard name, never cached). Timeline: Wi-Fi off at boot, enabled at 40 s, disabled and enabled at 100 and 110 s, W5500
PHY powered off at 170 s (a software link down, no unplugging) and on at 250 s. Set the `first`/`second` substitutions
for both priority orders; `phy_boot: "false"` starts with the PHY down (a stand-in for no cable). Needs Wi-Fi and, for
the full matrix, a live Ethernet link. The DIAG lines carry no addresses, so captures can be quoted.
- The W5500 PHY driver's `pwrctl()` is a no-op; `dnsdiag::phy_power()` writes PHYCFGR (OPMD=1, OPMDC 110 off and 111
on, RST low then high) and pokes `phy->get_link()`. Power down alone is not reported as a link loss (traffic kept
flowing); the PHY reset at power on is, so the 250 s step is a short link drop and return.
- Harness trap: `lwip_getaddrinfo` on the loop task with DNS servers set and no interface up blocks until the DNS
timeout and trips the task watchdog; `lookup()` skips while offline.
- Counts on #22800 (lookups ok/failed, PR against its merge base): Ethernet first with link 66/1 vs 23/43, Wi-Fi first
with the PHY down until 250 s 57/0 vs 1/56, Ethernet first with the PHY down 58/0 vs 20/36. Dev empties DNS on every
Wi-Fi join with Ethernet as the default, and on the Ethernet lease with Wi-Fi as the default.
- While the PHY is powered down the W5500 MAC logs `send command timeout` every 2 s on any build (driver noise from
the test).
- Compare build sizes only between names of equal length: the name is stored several times, so two names that differ
in length move the flash delta by tens of bytes.
@@ -1,45 +0,0 @@
# nPM1300 PMIC on the Seeed XIAO nRF54LM20A (#22661)
Scripts: `scripts/npm1300/`. The board's nPM1300 sits at 0x6B on P1.18 (SDA) and P1.17 (SCL); board facts, including
which rails must never be switched off, are in `reference/boards.md`.
- Probe first: `npmprobe.yaml` has the hub with no regulator or charger settings, scans the bus and dumps the
registers every 20 s, so you see what is on before any setting is written.
- `npm-all.yaml` (with `npmcommon.yaml`) has every sensor, binary sensor, text sensor and switch, logs each value from
`on_value` and `on_state` lambdas (entity state lines are VERBOSE on dev, so DEBUG shows nothing without them), dumps
the registers and switch states, and toggles the charger and LDO1. Put a setting under test (the VBUS limit, the
charger values) in `npmcommon.yaml`, not in the per-run file.
- `regdump.yaml` reads a fixed register list with `id(pmic).read_register16()` and logs `REGS 0201=.. ...`; compare two
builds' dumps after stripping timestamps and source line numbers.
- `runone.sh NAME CONFIG PORT [SECS]` flashes with `esphome run --no-logs` (pyOCD) while a serial capture stays open on
the same port. Builds without `ota:` have no MCUboot.
Findings:
- With no battery and the charger on, VBAT reads about 4.23 V, IBAT about 0.2 mA, battery detected is ON and the status
flips between Trickle Charge and Constant Current; with the charger off it says No Battery, and it can drop to Not
Charging for one update on its own. The battery temperature stays at 24.6 C (a fixed 10 k on the NTC pin). Charge
current scale and sign need a real cell.
- ADCIBATMEASSTATUS (0x0510) reads 0x0C (charge) most of the time with no cell and the charger on, sometimes 0x08 or
0x0F.
- `vbus_current_limit: 300mA` writes ILIM0 and ILIMSTARTUP as 0x03 and the board stays up on USB.
- BUCKSWCTRLSEL (0x040F) keeps its value across reflashes until a power cycle, so a setup write to it shows no change on
a warm board.
- To prove a setup priority change, add `on_boot` hooks at 1001, 950 and 900 that log markers and see where the
component's first publishes land between them.
## Charger defaults (#22769)
`npmhead.yaml` runs the register dump, the switch dump and the toggles once, 8 s after boot, on top of
`npmcommon.yaml`; put the charger block under test in `npmcommon.yaml`.
- nPM1300 charger registers with the reset-value defaults (32 mA, 3.6 V, 10 %, 2.9 V, 1000 mA; #22769): 0x0308/09 =
08/00, 0x030A/0B = CF/01, 0x030C = 0x030D = 02, 0x030E = 0x030F = 00. BCHGDISABLESET (0x0306) reads back 0x02 when
the NTC check is off (`thermistor_resistance: none`, ADCNTCRSEL 0x050A = 0) and 0x00 with 10k.
- No cell, charger on: BCHGCHARGESTATUS 0x05 (bit 0 plus trickle), VBAT about 3.61 V with VTERM 3.6 V (4.23 V with
4.2 V), 0.02 V with the charger off. NTCSTATUS (0x0332) and DIETEMPSTATUS (0x0333) read 00 throughout, so the NTC
hot and cold states cannot be reached on this board. BCHGERRREASON stays 00 across on/off toggles.
- A bare `charger:` key fails validation ("expected a dictionary"); use `charger: {}`. `esphome compile
--only-generate` with a private PLATFORMIO_CORE_DIR is enough to check the generated NPM1300Config initializer
without a board; a fresh private PlatformIO dir builds four nRF54 configs in about 10 min. A capture started before
the flash holds the old firmware first; cut at the last `Log initialized`.
@@ -101,14 +101,3 @@ once on a build without TCP, or `tcp ok udp ok` twice and no error with it. Add
come back, and connect with `apicheck.py ADDR` over Thread. Measured on the ItsyBitsy: 12.9 KB less flash and 17.1 KB
less RAM without TCP (Zephyr's default pools, 16/16 instead of 24/48), and SMP over UDP OTA still works with them.
Results: https://github.com/esphome/esphome/pull/21910#issuecomment-6518287139
## sdk-nrf 3.4.1 on the ItsyBitsy (#22233, #22558)
- Set `framework: version: 3.4.1` and `board: adafruit_itsybitsy`.
- Trees before #22233 include `<nrf.h>` in `logger_zephyr.cpp`, which 3.4.1 no longer has on the include path, so
Adafruit bootloader boards fail to build (`nrf.h: No such file or directory`). To test such a tree on 3.4.1, change
that include to `<nrfx.h>` locally.
- On the new USB stack, `usb_cdc_acm` bulk echo ran at about half the legacy rate (67 against 121 KiB/s for 64 KiB),
and `flush()` returned `3`. The logger, DTR, line coding and the 1200 baud touch worked.
- The zephyr_ble_server flow above, on 3.4.1: before #22558 advertising failed to restart after the first disconnect
(`Advertising failed to start (rc -12)`); with it the server took 39 connections in 120 s.
@@ -1,32 +0,0 @@
# one_wire
## ESP8266 interrupt locks under Wi-Fi load (#22452, #22453)
Code that holds interrupts off for long on the ESP8266 starves the Wi-Fi interrupt handler (reset signatures in
`reference/boards.md`). `scripts/esp8266_lock_load/` is the shared harness for any component that takes such locks:
- `base.yaml` is the shared DUT base: reset reason and info logged at boot, `power_save_mode: none`, `web_server` for
the HTTP load, `api: reboot_timeout: 0s`.
- `one-wire.yaml` reads a DS18B20 on GPIO4 every 2 s and runs a bus search every `search_interval`; `busy_control:
"true"` replaces the search with the same busy time (13 ms) and no 1-Wire traffic.
- `ir-one-pin.yaml` is the remote_transmitter case (see [remote_base.md](remote_base.md)).
- `wifi_load.py HOST SECONDS PPS` sends 512 byte UDP datagrams to port 9 at a fixed rate plus back to back GETs of
the web_server page.
- `run.sh` builds and flashes a DUT, captures the serial log, runs the load and prints every reset reason (usage in its
header). A clean run has one BOOT line.
Make one file per build with `substitutions: {name: ...}` and `packages: {t: !include one-wire.yaml}`, with
`secrets.yaml` next to it, and give dev and the PR different names.
Notes:
- Run each build for 45 minutes at 500 packets per second. Shorter or lighter runs often show no reset on either
build: one search every 5 s survived on both.
- The busy-wait control separates the lock from the CPU time. On #22452 it had no Hardware Watchdog in 45 minutes,
while the PR still had 2 (dev had 6), so the remaining resets came from the slot locks. All three builds had one
Software Watchdog, which is the busy time plus the load.
- A search with no sensor answering is much shorter than one with a sensor (about 190 slot locks per search with
one DS18B20), so test with a real sensor on the bus.
- A stale `secrets.yaml` gives `No networks found` and the run has no IP to load.
- The 1-Wire bus code is shared by every platform, so check a lock change on an ESP32 too: the same DS18B20 search
config on ESP-IDF with `bluetooth_proxy` scanning and the same load gave 0 resets and no missed searches on dev and
on #22452.
@@ -47,6 +47,7 @@ each ring size at least the encoder window, three times, and logs the best time
`BENCHDONE`. `bench.sh CONFIG DEVICE` builds and uploads `bench-esp32s3.yaml` or `bench-bk72xx.yaml`, collects one pass
over syslog and fails when a case did not reach the end of its stream.
- LibreTiny defines `micros` as a macro, so the header calls `(esphome::micros)()`.
- Keep the streams near 130 KB on a BK7231N: an image with about 330 KB of extra incompressible data failed its
OTA with "Writing OTA data to flash memory failed". Use one image and `--windows 12` there.
- `bench.hpp` targets the decoder interface with `dest_limit`. #21641 drops it (the output runs into the window up
@@ -83,34 +84,3 @@ case's fires in order with progress collapsed to a count; diff dev against the P
- A random file through web_server gives `error 128` on the ESP32 and `error 132` on the ESP8266.
- A SIGTERM to the uploader lets it finish the transfer; use SIGKILL for the native error path.
- Results: https://github.com/esphome/esphome/pull/21903#issuecomment-6516529409
## Signed OTA with several trusted keys (`verification_keys`, #22398)
`scripts/signed_ota/` on an ESP32-C3 (`signed-ota-c3.yaml`, `signed_ota_verification: signing_scheme: rsa3072,
verification_keys: [keyA.pem]`):
1. `python -m espsecure generate-signing-key --version 2 --scheme rsa3072 keyA.pem` (and keyB).
2. Build four tagged images with `esphome -s tag <name> compile`, serial-flash the first factory image (no on-boot
signature check without secure boot), copy each `firmware.ota.bin`.
3. `sign_variants.py <image> keyA.pem keyB.pem <prefix>` writes key A, key B, B then A, stored CRC flipped, byte 2
changed (covered only by the CRC), the same with the CRC recomputed by zlib, and B then A with block 1's CRC flipped.
4. `otaseq.sh <ip> FILE=reject... FILE=<tag>...` sends them with `esphome upload --file` and checks the first verdict
in the device log and the upload result against the expectation (a rejection, or the `BUILDTAG` of that build);
send the rejects first so every check runs in the same build, then the accepted ones.
Expected: unsigned, B only and every CRC edit rejected; A, B+A and the zlib-fixed CRC accepted. The byte 2 pair isolates
the CRC from the signature and key digest.
Gotchas: an ESP-IDF image can contain `E7 02` at a 4 KiB boundary by chance; find the signature sector by the CRC, as
`sign_variants.py` does. An unsigned image sent after a signed one can be rejected with "trusted key failed to verify"
instead of "no signature block": the partition still holds the old signature sector past the new image's end.
Results: https://github.com/esphome/esphome/pull/22398#issuecomment-6716330334
## ESP8266 web_server OTA interruption and SSE (#22682)
`scripts/web_server/async_bump/otacases.py` (README there). Facts from a D1 mini on dev and on ESPAsyncWebServer 3.12.1:
- A cut upload logs `OTA upload interrupted; aborting`, keeps the old image and serves again within a second; the
next full upload works without waiting for a lock release. No heap lost per cut.
- Uploads stop by themselves on the network about 1 in 6 to 1 in 15 (curl `000`, old image kept), and about 1 in 25
boots the new image while curl never gets the reply (it waits 30 to 60 s). Both builds; classify, do not fail on it.
- At OTA reboot the device does not close `/events`: a held client sees silence, rarely a FIN. Detect by idle time.
- Max free block drops from about 45 kB to about 37 kB after the first OTA and stays (both builds).
@@ -1,15 +0,0 @@
# Per period PWM capture on the same chip (#15438)
`scripts/pwm_capture/` (ESP32-C6): `pwmcap.hpp` routes the `ledc` pad through the GPIO matrix into RMT receive at
80 MHz (ping pong buffers, `en_partial_rx`, `mem_block_symbols: 96`) and a PCNT unit, checks every period in the IRAM
safe RMT interrupt and logs a `WIN` line every 10 s (high time and period histograms, minimum and maximum, level
changes, spikes, reversals). `common.yaml` needs `include_builtin_idf_components: [esp_driver_rmt, esp_driver_pcnt]`
and `CONFIG_RMT_ISR_IRAM_SAFE` and `CONFIG_PCNT_ISR_IRAM_SAFE`; build it with `-s name NAME -s ps light` (or `none`).
`run.sh NAME SECONDS [--every 5 --transition 1]` (env `TREE`, `PORT`, `SC`, `PY`) resets and captures while
`apiclient.py` holds an API connection and optionally sends light transitions; `summ.py` totals the windows.
- LEDC and RMT share PLL_F80M, so a 20 kHz 11 bit output reads exactly 4000 ticks per period with no jitter.
- Result: no glitch in about 10 million periods with power save `light` or `none`; #15718 (LEDC in IRAM) changed
nothing. Positive control: a lambda that changes the duty for a few periods is caught every time.
- Window boundaries follow RMT chunks (about 1000 periods), so `n` alternates between 199584 and 200592; sum over
the run.
@@ -12,8 +12,7 @@ markers and prints what each sent frame decoded as.
- ESP32: `remote_transmitter` and two `remote_receiver`s on the same GPIO (`allow_other_uses`), carrier 100%, no
inversion. RMT reads its own output, every frame decodes. Second receiver `dump: [nec, raw]` shows raw runs only
where no primary matched.
- ESP8266 before #22453: the transmitter sends the whole frame with interrupts off, so a receiver on the same pin
sees nothing (from #22453 on it decodes, see below). For older trees, `softtx.hpp`
- ESP8266: the transmitter sends with interrupts off, so a receiver on the same pin sees nothing. `softtx.hpp`
bit-bangs frames built with each protocol's `encode()` with interrupts on, and an `interval` sets the pin back to
OUTPUT after the receiver's setup. Bit-banged NEC decodes as NEC about 2 of 3 times (timing), on every build.
- Dish never decodes in loopback on dev either. Pronto matches every frame, so raw on `dump: all` never prints.
@@ -29,18 +28,3 @@ Results: https://github.com/esphome/esphome/pull/21563#issuecomment-6381304387
Retest of the moved stack with 5 rounds per build: https://github.com/esphome/esphome/pull/21563#issuecomment-6382075726.
A D1 mini CH340 sometimes fails the 460800 baud upload; esphome retries at 115200 by itself, but a runner that reads
only the last line sees "Failed to upload" first, so rerun before blaming the build.
## ESP8266 transmitter to receiver on the same board (#22453)
- Wire the `remote_transmitter` pin to a `remote_receiver` pin (GPIO12 to GPIO14 on a D1 mini), `inverted: false`.
- Keep the receiver's default `idle`. `idle: 4ms` is shorter than the NEC leader space (4.5 ms), so every frame is
split and nothing decodes.
- Dev before #22453 sends the whole frame inside one `InterruptLock`, so a receiver on the same chip decodes 0
frames. That is expected on old builds, not a wiring fault. With locks per edge, 3,692 of 3,695 frames decoded
under Wi-Fi load.
- Count with `on_nec` matching a counter in the command, and log the counts on an `interval`.
- Without a jumper, put both on GPIO12 with `allow_other_uses: true` and call `pinMode(12, OUTPUT)` before each frame
(the receiver's setup leaves the shared pin as an input); the receiver's edge interrupt sees the transmitter's own
output. `scripts/esp8266_lock_load/ir-one-pin.yaml` does this, and its `run.sh` adds Wi-Fi load (see
[one_wire.md](one_wire.md)). Under load, the build that sends the whole frame in one lock hit a Hardware Watchdog in
`wDev_ProcessFiq` after 11 minutes.
@@ -21,19 +21,3 @@ captures 40 s from boot and runs `ivsum.py`, which prints the run count, first r
runs), so do not compare those offsets between builds.
- `main.cpp` shows the class each interval gets (`IntervalTrigger` or `DelayedIntervalTrigger` on #21799).
- Results: https://github.com/esphome/esphome/pull/21799#issuecomment-6516798398
## Freelist trim after boot (ratgdo/esphome-ratgdo#705)
10 s after setup the scheduler frees its item pool and shrinks its `to_add_` and `items_` vectors
(`SCHEDULER_FREELIST_TRIM_DELAY_MS` in `application.cpp`). The first burst of timers after that allocates new
items and regrows the vectors once, so heap or timing measurements should skip the first run after boot.
## RP2 scheduler lock under IRQ and lwIP load (#22688)
`scripts/rp2_sched_lock/` (README there). The `irq_sched_stress` external component calls `defer()` and `set_timeout()`
from a 300 us pico-sdk repeating timer while the main loop does the same; the removed item race integration fixture
runs on the board as an API action. `flow.sh` runs the actions, API connect churn (`../api/connstress.py`), REST POST
churn and both together, then the actions again, and fails on a failed action, a failed connstress 500x2 or a reboot.
Judge by counts, not order: RP2 is `ESPHOME_THREAD_SINGLE`, so `defer()` order is not FIFO on any build. Keep
`debug_scheduler` off, it logs from the IRQ. Dev before #22688 failed the defer count on the Pico W; the PR passed.
`looptime.sh` compares the 16 ms interval gaps with the API and page loads running between two builds.
@@ -1,23 +0,0 @@
# serial_proxy with a simulated device (#22687)
`scripts/serial_proxy/`: no external device. On an ESP32-C3 (logger on USB_SERIAL_JTAG) the proxy UART is tx 4 rx 5
and the simulator UART tx 5 rx 4 (GPIO matrix, `allow_other_uses` on all four pins). `spsim.hpp` is the device; the
client drives it through the proxy with 7E-framed commands: `R` one 88 byte reply (A5, seq u32 BE, 83 bytes
(seq + j) % 251), `S` stream replies back to back, `A` ASH DATA frames every 50 ms with ACK latency timing (for
`ezsp_proxy_tap` in PROTOCOL mode), `X` log `SIMDONE` totals and reset, `B<baud>` change the sim's baud. SIMSTAT and
ASHWIN lines each second carry `backlog_max`, the proxy UART's RX backlog (bytes held back). The sim polls every
16 ms, the main loop cadence (a 1 ms interval also speeds up the proxy's loop and skews what it measures); its 1024
byte TX buffer holds the 185 bytes 115200 baud needs per poll, and ACK latencies are good to one poll.
- `spclient.py HOST --baud B --mode reply|stream|ash [--interval MS] [--duration S | --pauses N --pause S --read S]
[--protocol]`: raw plaintext API client (no encryption in the config), subscribes, brings the sim and the proxy to
the baud, pauses reads on a schedule and prints a RESULT line (complete and broken replies, received bytes). Compare
`received` with the sim's `SIMDONE bytes`.
- `suite.sh BUILD TREE` (env PORT) flashes `spBUILD.yaml` (a copy of `example-run.yaml` with its own `name`),
captures, runs every case (`CASES="a b"` picks some).
- `rx_buffer_size` on the proxy UART decides how long a blocked client can stall before the UART overflows
(32768 used; at 115200 a 5 s pause peaks near it).
- macOS ignores a small SO_RCVBUF after connect (reads back 36000), so a pause needs a few seconds to back up.
- Dev at 1920 baud loses bytes even with the client reading: the proxy sends 1 byte API messages (the UART wakes
the loop per byte) and the TCP queue fills; #22687's wait coalesces them. A tap in PROTOCOL mode keeps that loss.
- Stream mode at 115200 (11.5 KB/s) is more than the proxy forwards over Wi-Fi; both builds lose there.
@@ -73,16 +73,3 @@ address they do not simulate.
- An `addressable_light` display writes nothing until its light is turned on.
- Before calling a rebased PR's change trivial, diff the PR's own diff (head against its parent) old against new: a
move that looks like a merge only can carry real commits from below.
## TRF7962A fake chip (#22188)
`scripts/trf7962a_sim/`: `trfsim.hpp` replaces the SPI component's bus (`SPIComponent` is final, so `spi_bus_` is
reached through the explicit-instantiation member pointer trick) with a `FakeBus` from `on_boot` priority 800, between
the bus (1000) and the reader's setup (600). The bus hands both of the reader's clients (MODE0 write, MODE1 read)
delegates on one register, FIFO and IRQ model; the IRQ pad is switched to input/output after setup and driven by the
model. `trfpr.yaml` runs the tag scenarios (none, present, slow, flaky, swap, collision, CRC, error flag, bad MSB,
TX only, stuck IRQ) and prints per-phase counters (CS, field timing, TX payload, reads without an address). The
same FakeBus approach works for any SPI chip whose reply depends on the address byte inside one CS window, which the
ESP32 SPI slave cannot fake.
The header targets the SPI API of the #22188 branch (`SPIBus` and `SPIComponent::spi_bus_`); dev has since dropped
`SPIBus`, so on a newer tree the bus hook has to move to the new delegate creation before it builds.
@@ -1,32 +0,0 @@
# SPI bus capture on the same board, no chips or wiring (#22376)
`scripts/spi_self/`: `gen.py` writes `spi<board>.yaml` (e32 classic ESP32, c3, e66 ESP8266, rp Pico 2 W) with a
hardware bus, a software bus, several `spi_device` instances (all four modes, MSB and LSB first, different rates and
CS pins) and a `max7219`. `spitest.hpp` drives every device from a 20 ms interval (an 8 byte transfer, write16, a 12
bit write, read_array, write_array, back to back transfers, teardown plus setup, a second setup, a max7219 update),
three passes, then reboots. `run.sh BOARD LABEL SECONDS` (env `TREE`, `PORT`, `SC`, `PY`, and `PORTS` for the Pico)
builds, flashes, captures and checks; `analyze.py spi<board>.yaml BASE.txt PR.txt` checks each record against the
expected wire data and diffs the two builds by the majority payload per record, so interrupt jitter does not count.
- Classic ESP32: the ESPHome bus is on SPI2 through the matrix and SPI3 runs as a slave without DMA (64 bytes at
most), set up again in the device's mode for each operation. Its nominal CS must be a spare pad held high
(GPIO14); the device CS is routed into `spics_in` only after `spi_slave_queue_trans()`, because the slave driver
reconnects the nominal pad when a transaction loads. Never give the slave the device's CS pad:
`spicommon_cs_initialize` takes over its output and `spi_slave_free` resets it. `gpio_set_direction(..., OUTPUT)`
turns a pad back into a plain GPIO output, so only call `gpio_input_enable` on pads the master drives.
- With that slave attached, full duplex reads on the master lose their last bytes after the first read of a boot,
even through a MOSI to MISO loopback; a config without the slave reads fine. Do not judge reads on that layout.
- The slave sees bit-banged mode 1 data one bit late; the interrupt captures decode it correctly.
- C3, ESP8266 and Pico 2 W: GPIO edge interrupts on SCK and CS take snapshots of the input register. C3 SPI2 cannot
clock below about 78 kHz; at 80 to 100 kHz the capture works with
`gpio_install_isr_service(ESP_INTR_FLAG_LEVEL3 | ESP_INTR_FLAG_IRAM)`, but one build lost an SCK pair at the start
of a frame in every run and a rebuild did not: compare two builds before calling a missing edge real. Let the log
output drain (20 ms) before a capture that follows `spi_setup()`.
- ESP8266: the HSPI pads (12, 13, 14) do not show in `GPI` while in SPI function; the script logs `SPI1CLK`,
`SPI1C`, `SPI1U` and `SPI1P` after each hardware bus operation instead.
- Pico 2 W: `IO_BANK0` INOVER on the MISO pad forces the SPI receive input low or high (reads 00 and ff).
- Software bus loopback: MOSI and MISO on one pad with `allow_other_uses`, then set the pad back to output.
- Seen on dev before #22376: the software bus ignores `data_rate` (`cycle_clock_()` never waits), and SCK moves to
the new idle level only after CS is low when devices with different clock polarity share the software bus.
- LibreTiny (BK72xx, RTL87xx, LN882x) has no `spi:` (config validation allows esp32, esp8266, nrf52 and rp2 only),
so SPI changes cannot be run there.
@@ -72,23 +72,3 @@ case, and `T1 X` shows which frames got through.
about 49 s after reset. Only the second boot logs `CASE done`, so a skipped reboot fails the run.
See `rs485_frame.md` for the same cross-wired UART trick on an ESP32-C3.
## Duco ventilation box (#22560)
`scripts/uart_sim/duco/`: `ducosim.hpp` is the box model from `tests/integration/fixtures/uart_mock_duco.yaml` (ack
plus answer, AA 01 stuffing, CRC16) on a second UART of the same board; on an ESP32-C6 the hub used tx 18 rx 19 and
the box tx 19 rx 18 at 57600, wired across. `gen.py NAME pr|dev` writes the config (every text sensor, sensors, mode,
comfort, sntp clock) with an `on_value` log per entity (`STATE id = '...'`, newlines as `\n`); `DUCO_VERBOSE=1` shows
`Found node`, `DUCO_SERIAL_SIZE=18` gives the dev fixture's short serial answer. `run.sh NAME KIND SECONDS` (env
`TREE`, `DUCO_PORT`, `PY`, optional `BUILD_LOCK`), run from a scratch directory with a copy of `ducosim.hpp`,
generates, builds through `scripts/pio_build.sh`, flashes and captures with `scripts/serial_capture.py`; a pr build
ends with `check.py` against `expected.txt` (exit 1 on a mismatch), and the model logs requests it has no answer for.
- Answer one request per poll call. Answering a 26 request burst in one pass blocks about 100 ms and overflows the
hub's 256 byte UART RX buffer, which silently drops the serial and version answers every round.
- Dev's hub frame is 32 bytes: the PR model's 34 byte serial answer is dropped there and duco sets "No answer".
- Error cases 7 (three long, 210 B) and 8 (four short, 148 B) check that the code-only fallback goes by length.
## ESP8266 software serial byte checks
The worst case for an unlock after a trailing run of high bits is a frame of bytes all below 0x80; a mixed frame
hides it.
@@ -24,8 +24,3 @@ identical payloads share one table, a lambda is sent from its vector. An empty c
(`sendto() error 22` in the log, no datagram) until it accepted `nullptr` with a zero length; the old
`TemplatableBytes::visit` had passed a zero length stack buffer instead. Compare sizes against the stacked PR's own
head when the PR is stacked, not against dev. Results: https://github.com/esphome/esphome/pull/21263
## Two sockets on one port (#22509)
lwIP lets two sockets bind one UDP port when `SO_REUSEADDR` is set, and unicast then goes to the socket opened
last; without it the second bind fails with errno 112.
@@ -59,66 +59,3 @@ valve, update, alarm_control_panel or water_heater entity reboots with `Exceptio
`json::linked(const char *)` (the ROM `strlen` on a flash address), fixed by #21316. The cover in the template keeps
that path covered; before #21316 is merged, apply its one line change to both trees or the web_server comparison
cannot run.
## Basic auth on the async web servers (ESP8266, RP2040, LibreTiny)
`scripts/web_auth/` (see its README) fuzzes Basic auth over raw sockets: functional routes with and without
credentials, wrong credentials, other schemes, a tampered hash, spacing and case, then large headers, many failed
attempts next to an authenticated `/events` stream, and a timing comparison of wrong first and last bytes. It also
has a captive portal client package that probes `/`, `/config.json`, POST with and without credentials and
`/wifisave`. `compare.py` lines a dev run up with a PR run. These scripts were still being extended when copied here.
The README also covers the newer flows: the heap under paced connection churn (`heap_churn.py`, the LibreTiny TIME_WAIT
check for #22696) and the main task stack high-water mark with the JSON arena (`arena/`, #22699).
## Per-session /events listing on ESP32 (#22691)
Checks that a new `/events` client's entity listing goes only to that client. Config
`scripts/web_server/sse-listing.yaml` (ESP32 ESP-IDF, web_server version 3, 19 entities, 3 sorting groups, heap logged
every 5 s); build each head with a different `-s name` through `pio_build.sh`. Driver:
`python -I scripts/web_server/sse_listing.py DEVICE_IP --rounds 4 --out events.jsonl`. Each round opens streams 1 and
2, waits for their listing, connects stream 3 and counts for 10 s per stream. A `state` event that carries `domain`
is a full detail (detail_all) entry; a `ping` with only `uptime` is the 10 s keepalive, any other `ping` the greeting
(config JSON). The greeting and `sorting_group` events always go only to the new session, so only the detail_all
count tells the builds apart. Ends `RESULT PASS` or `RESULT FAIL` (exit 0 or 1).
On an ESP32-PICO-D4 the control sent all 19 detail_all entries to streams 1 and 2 on every round, the PR none; stream 3
was ordered on both.
## /events stream rework checks (#21973)
Config `scripts/web_server/stream-dut.yaml` (ESP32 ESP-IDF, 20 ms template sensor, a "Wifi Drop" button that runs
`wifi.disable`, 8 s, `wifi.enable`, heap every 5 s). `scripts/web_server/stream_retest.py HOST MODE` has five modes:
`head` (raw bytes of the first 3 s; compare two builds with `head_compare.py A B NAME_A NAME_B`, which masks numbers
and the device name), `multi` (4 clients 20 s, events and longest gap), `stall` (a client with a 2 KB receive buffer
stops reading for 35 s; read the close time from `Closing stuck stream`, or `Closing stuck EventSource connection` on
older builds), `wifidrop` (REST press, the client reconnects after 5 s idle like a browser `EventSource`) and `churn`
(40 connects, 20 of them reset, then a fresh client). `stream_runall.sh NAME PORT` uploads, captures and runs every
mode. Both builds should close the stalled stream at about 20 s and close the stream at the Wi-Fi drop.
- `sse_listing.py` prints stream 3's TCP connect and first event times. A late greeting (3 to 4 s) was TCP connect
time on the network (SYN retransmits), seen on the control too, not the device.
- On a noisy Wi-Fi network multi-client runs fell to 110 to 900 events per client with every client pausing together,
and a stalled client sometimes missed the EOF although the device logged `Closing stuck stream`. Alternate the
builds (flash A, 4 multi runs, flash B, repeat) before calling a rate difference.
- Log text check for refactors: collect every `[web_server*]` line from all captures, mask timestamps, source line
numbers and digits, `sort -u`, and diff the two builds' sets.
## ESPAsyncWebServer / AsyncTCP bump A/B (#22682)
Scripts: `scripts/web_server/async_bump/` (README there). ESP32 never uses ESPAsyncWebServer: `web_server_base` loads
`web_server_idf` for every ESP32 build, Arduino included, so only ESP8266, RP2040/RP2350 and LibreTiny need a run.
Per board: page loads, a 180 s SSE client, REST control, `/update` OTA with the build's own image, restart button,
then the captive portal hammer (`scripts/wifisave_hammer.sh`, `bogus_saves: "5"`) on one board per family.
- Library check: ESP8266 builds use the native toolchain; the library copies sit in
`.esphome/pio_components/arduino8266/<hash>/esp32async/*/library.json`, not in `.piolibdeps`.
- Restart button over REST on LibreTiny: the HTTP reply is often lost (curl 000 after 5 to 10 s) although the board
restarts, on dev too. Not a regression signal.
- LibreTiny AsyncTCP 3.5.0 does not build with `framework: loglevel: debug` (or more verbose): `AsyncTCP.cpp:332` logs
`xPortGetCoreID()`, undeclared on LibreTiny; `log_d` is empty at the default level, so normal builds pass. Any
library bump for LibreTiny needs one build with the framework loglevel at debug.
- SSE rates on a board with a weak radio (a CB3S, say) are unreliable; decide SSE questions on a board with a good
link and the same SDK.
- `/events` churn (`rp2_web_ble/churn.sh`) faults `AsyncEventSource::send` in `AsyncClient::connected()` from
`deferrable_send_state` on the Pico W and the W5500-EVB-Pico; 3.12.1 and the RPAsyncTCP fork do not fix it.
- Stalled-reader runs need about 150 s: the client's receive window can take about 60 s to close, so a 60 s stall run
ends before the 20 s device timeout even starts.
@@ -93,94 +93,3 @@ connected channel starts and ends on the bench channel with another channel in b
- Dropping the AP under a connected device gives a `Beacon Timeout` link loss.
- The 90 s interval fires once right after boot.
- A `Disconnected ... Association Leave` right after `Connecting to` is the aborted attempt, not a missed connect.
## Soft AP with the BLE tracker scanning (#22534, #22566)
`scripts/ble_scan_soft_ap/`: `dut.yaml` runs only `wifi: ap:` with `captive_portal:` and `bluetooth_proxy:`, and
`client.yaml` (on a second ESP32) joins that AP and fetches the portal page every 5 s, logging `JOIN` and `STAT`
lines. `run.sh` erases and flashes the DUT, resets the client and prints the generated scan window, the first join, the
page counts and the client's disconnect reasons (usage in its header).
- A scan window equal to the scan interval (320 ms of 320 ms) leaves the radio no time for the soft AP: on dev no
client joined on the classic ESP32, S3, C3 or C6. With #22566 the tracker uses its 30 ms connection window while
the AP runs, and every chip took a client in about 4 s.
- Only one AP with the test SSID may be up at a time; list the other AP boards in `PARK` so they wait in their ROM
bootloader.
- To check that the full window comes back after provisioning, log `esp32_ble_tracker` at VERBOSE and provision
with `templates/client-wifisave.yaml` (set `ap_ssid` to the DUT's and `ap_password: ""`, the DUT's AP is open): `Connection or AP active, using 48 unit scan window` must stop once the AP stops.
- Some clients do not join an ESP8266 AP at all (the C6 failed on dev too with `Too Many Associations`); use a C3.
## ESP8266 power save after captive portal provisioning (#22542, #22564)
The DUT runs `ap:` plus `captive_portal:` with no station network and logs `wifi_get_sleep_type()` every 5 s (0 is
`NONE_SLEEP_T`, 2 is `MODEM_SLEEP_T`); an ESP32-C3 with `templates/client-wifisave.yaml` posts the real credentials.
Erase the flash before each run so no saved credentials skip the portal. On dev the sleep type stayed 2 after
provisioning although the config asked for none; with #22564 it read 0 from the moment the AP started, and the
`networks:` path still read 0.
## Counting the Disassociation burst at reboot (#22715)
`scripts/wpa3_sniff/`: `sniffap.yaml` is a WPA3-SAE only AP with PMF required (any ESP32, set `board:`) that also turns
on promiscuous mode with the management filter (`sniff_inc.hpp`) and logs each assoc request, auth, disassoc and deauth
frame with transmitter, sequence number, Retry and Protected bits. The AP's own sniffer does see retransmitted copies
addressed to it. The station configs share `st-base.yaml`: `st-common.yaml` reboots 10 s after connect (alternating
`App.reboot()`/`safe_reboot()`), `stearly-common.yaml` reboots on the first loop pass after `WIFI_EVENT_STA_CONNECTED`
(the event handler wakes the loop and `on_loop` runs before the Wi-Fi component's loop; measured 0.15 to 0.46 ms
after the event with a 1 ms interval, before the Wi-Fi component handles it), `ds-common.yaml` deep sleeps 5 s.
`run.sh BUILD TREE SECONDS` (env ST, AP) flashes the station and captures both; `an15.py BUILD STA_MAC` groups the
copies per sequence number within one burst (copies more than 2 s apart start a new group).
- On ESP-IDF 5.5.6 the ESP32-C6 station sends 27 to 32 protected Disassociation copies on every reboot while
associated (espressif/esp-idf#19443); the classic ESP32 sends 1. Use a C6 (or C3) as the station.
- Deep sleep on dev sends no Disassociation at all; the AP finds out only when the station returns (reason 2).
- `esp_wifi_sta_get_ap_info()` already returns OK during the SAE exchange, before association completes.
- A capture over the C6 USB-JTAG console loses the last log line before deep sleep; count from the AP side.
- Serial capture timestamps are quantized to about 200 ms on a CH340, too coarse to time a 100 ms shutdown wait.
## Configured credentials from flash (#22757)
- Flows: two `networks:` with a hidden bogus one at higher priority, then the real one; the `wifisave_hammer.sh` flow
with `bogus_saves: 3`, long AP SSID and password (over 18 characters, the `CompactString` inline capacity), then an
RTS reset and an API restart; `wifi.configure` from `on_boot` after 25 s if not connected, then a reset (prints
"skipped"); a lambda building `WiFiAP` from runtime `std::string` and calling `set_sta()` + `start_connecting()`.
- The ESP-IDF API client sees `Connection reset by peer` on disconnect after a button that reconnects Wi-Fi; harmless.
## ESP32 with a hidden network first never joins (#22764)
- On dev an ESP32 with a hidden bogus network listed first ends every attempt on the real network at once with
"Unspecified" or "Auth Leave" and never joins; without `hidden: true` it joins. An ESP8266 joins either way.
- Cause, confirmed with debug prints: after the EXPLICIT_HIDDEN probe fails, the first real attempt calls
`esp_wifi_disconnect()` (config changed, not associated, nothing marked expected). The hidden attempt's leave event
(reason 36 STA_LEAVING, shown as "Unspecified", the bogus SSID, a zero BSSID) is then charged to the real attempt.
Attempt 2 gets reason 36 too, and the adapter restart aborts its auth (reason 3 "Auth Leave"). This loops forever on
ESP-IDF 5.5.6, with the esp-idf and arduino frameworks alike.
- ESPHome does not print numeric reasons; add one log line in the STA_DISCONNECTED handler and one before
`esp_wifi_set_config` to see them. `esp32: framework: log_level: INFO` shows the IDF `state: init -> auth` lines.
- Regression check: 2025.10.5 joins the hidden-first config (no explicit hidden phase); 2026.10.2 and dev loop.
- Soft AP outage on demand without touching the real AP: a D1 mini AP-only config with `enable_on_boot: false` and an
`on_boot` delay of 20 s before `wifi.enable`; an RTS reset of the D1 gives a 20+ s outage (an ESP32 station sees
Beacon Timeout). The ESP8266 soft AP DHCP can take about 20 s to answer after association.
- A soft AP that still holds an ESP32 from its previous boot rejects the first association with 'Too Many
Associations' (dev too), and some APs reject the first auth after a reset ("Authentication Failed" about 10 ms in,
attempt 2 joins).
## Router outages timed into the portal cooldown (#22797)
`scripts/wifi_outage/`: `rtr.yaml` makes a D1 mini an AP-only test router (its own SSID and password from secrets
`t_ssid`/`t_pw`) that logs `AP client connected/disconnected`, `AP Distribute Station IP` and a 1 s `RTR` station list.
`dut-common.yaml` is a BK7238 DUT (one network, `ap:` with `ap_timeout: 10s`, `captive_portal:`, UART1 log, `ALIVE`
line) with `dhcpwrap.hpp`, a `-Wl,--wrap=udp_sendto_if`/`udp_sendto_if_src` hook that counts DHCP DISCOVER, REQUEST
and RELEASE and logs `DHCPRELEASE sent n= up=` (`up` = netif still up). `c3-common.yaml` is an ESP-IDF C3 DUT without
AP. `outage.py DUT_PORT RTR_PORT LABEL N [--no-ap]` owns both ports: waits for 20 s of `conn=1`, holds the D1 in reset
(RTS on EN), waits for `→ RESTARTING` after `Starting fallback AP` (or `Restarting adapter` with `--no-ap`), lets the
router boot 1 s later and waits for the first line that ends the cooldown (the ones `an.py` uses, at most 40 s).
`an.py LABEL...` prints per outage skip, cooldown length, driver association and address inside the cooldown,
Connected-then-lost, held, releases after the router was back and router-side leaves.
- The D1 router is back about 1 s after release and hands out the address in about 0.2 s, but the BK7238 only reports
it 7 to 8 s after associating; the 30 s portal cooldown leaves plenty of room.
- BK7238 on dev: the Beken supplicant keeps trying after the skip branch's `WiFi.disconnect()` and joins about 5 s into
the cooldown on every outage; base then logs Connected and, 200 ms later, Connection lost (20/20), #22797 0/20. The
BK station never deauths on that teardown: the ESP8266 router logs `err already associed!` and only then the leave.
The release is issued with the netif already down (`up=0`), so it is not proof it hit the air.
- The first `Connection lost` in a capture is the reset from opening the D1 mini port (see `reference/boards.md`), not
an outage.
- About 2 min per outage; run blocks of 10 interleaved base/PR with a reflash between.
@@ -1,9 +0,0 @@
# wled
## Realtime sender on the bench (#22509)
When the access point stops passing traffic between clients (lwIP `sendto` on the sender fails with errno 12
because ARP never resolves), run the sender on a board instead of the host: an ESP32-S3 runs `wifi: ap:` and a
25 ms interval lambda sends WARLS, DRGB or DNRGB frames with lwIP `sendto` to the devices under test, which join
that AP with `manual_ip`. Two lights on the same port: see `udp.md`. Heap after an effect stops: see
`heap_alloc.md`.
@@ -21,28 +21,3 @@ uart_button/uartbtn*.yaml
value_range/rng*.yaml
ld2420_sim/ldenergy*.yaml
ld2420_sim/ldsimple*.yaml
apikey.txt
pio/
rp2_web_ble/*.log
rp2_web_ble/*.txt
rp2_web_ble/*.pcap
gatt_uuid/*.json
gatt_uuid/*.txt
web_auth/configs/*.log
web_auth/configs/*.txt
web_auth/configs/*.json
improv/*.log
npm1300/*-cap.txt
npm1300/*-cap.err
npm1300/*-run.log
rp2_sched_lock/*.txt
rp2_sched_lock/*.log
serial_proxy/*.txt
serial_proxy/*.log
web_server/async_bump/*.log
web_server/async_bump/*.txt
web_server/async_bump/*.pcap
web_server/async_bump/img-*.bin
wifi_outage/*.txt
wpa3_sniff/*.txt
api/overflow/putbe-sensors.yaml
@@ -1,88 +0,0 @@
"""
Rapid native API connect/disconnect stress test (from esphome#14687 and esphome#14706).
usage: connstress.py HOST [ITERATIONS] [CONCURRENCY] (default 500 iterations, 4 at a time)
env: API_KEY (base64 noise key; unset, empty or - for plaintext), API_PORT (default 6053), API_PASSWORD
Each attempt connects with login, disconnects and counts a failure on any error or a 10 s timeout on either step.
Exits 1 when any attempt failed. Watch the serial log for crashes: the client only sees resets.
"""
import asyncio
import contextlib
import os
import sys
import time
from aioesphomeapi import APIClient
HOST = sys.argv[1] if len(sys.argv) > 1 else sys.exit(__doc__)
PORT = int(os.environ.get("API_PORT", "6053"))
PASSWORD = os.environ.get("API_PASSWORD", "")
NOISE_PSK = (
None if os.environ.get("API_KEY") in (None, "", "-") else os.environ["API_KEY"]
)
async def connect_disconnect(client_id: int, iteration: int) -> tuple[int, bool, str]:
cli = APIClient(HOST, PORT, PASSWORD, noise_psk=NOISE_PSK)
try:
await asyncio.wait_for(cli.connect(login=True), timeout=10)
# a disconnect that hangs on a wedged device is a failure too, not a stuck slot
await asyncio.wait_for(cli.disconnect(), timeout=10)
return iteration, True, ""
except Exception as e: # noqa: BLE001 every failure is a counted result
return (
iteration,
False,
f"client{client_id} iter{iteration}: {type(e).__name__}: {e}",
)
finally:
with contextlib.suppress(Exception):
await asyncio.wait_for(cli.disconnect(force=True), timeout=10)
async def main() -> None:
iterations = int(sys.argv[2]) if len(sys.argv) > 2 else 500
concurrency = int(sys.argv[3]) if len(sys.argv) > 3 else 4
print(f"Iterations: {iterations}, Concurrency: {concurrency}", flush=True)
success = fail = 0
errors: list[str] = []
start = time.monotonic()
sem = asyncio.Semaphore(concurrency)
async def run(client_id: int, iteration: int) -> tuple[int, bool, str]:
async with sem:
return await connect_disconnect(client_id, iteration)
tasks = [asyncio.create_task(run(i % concurrency, i)) for i in range(iterations)]
for coro in asyncio.as_completed(tasks):
_, ok, err = await coro
if ok:
success += 1
else:
fail += 1
errors.append(err)
total = success + fail
if total % 10 == 0 or not ok:
rate = total / max(time.monotonic() - start, 1e-6)
print(
f"[{total}/{iterations}] ok={success} fail={fail} ({rate:.1f}/s)",
flush=True,
)
if err:
print(f" ERROR: {err}", flush=True)
elapsed = time.monotonic() - start
print(f"\nDone in {elapsed:.1f}s")
print(f"Success: {success}, Failed: {fail}, Rate: {iterations / elapsed:.1f}/s")
if errors:
print("\nLast 10 errors:")
for e in errors[-10:]:
print(f" {e}")
print("RESULT", "PASS" if fail == 0 else "FAIL")
sys.exit(1 if fail else 0)
if __name__ == "__main__":
asyncio.run(main())
@@ -1,173 +0,0 @@
"""apitest.py HOST OUT.json: Noise API check (entities, states, button) then a flood with a stalled reader
so the device's API overflow buffer fills and drains. Validates every received burst state."""
import asyncio
import gc
import json
import math
import pathlib
import socket
import sys
import time
from aioesphomeapi import SensorState, TextSensorState
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2]))
import aioesphomeapi.connection as conn_mod
from apiconn import connect # noqa: E402
HOST, OUT = sys.argv[1], sys.argv[2]
TL = 800
class SmallSock(socket.socket):
def setsockopt(self, level, opt, *a):
if level == socket.SOL_SOCKET and opt == socket.SO_RCVBUF and a[0] != 2048:
return None
return super().setsockopt(level, opt, *a)
async def start_small(addr_infos, **kw):
fam, typ, proto, _, addr = addr_infos[0]
s = SmallSock(fam, typ, proto)
s.setblocking(False)
s.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 2048)
await asyncio.get_running_loop().sock_connect(s, addr)
return s
conn_mod.aiohappyeyeballs.start_connection = start_small
def txt_ok(s, slot):
if len(s) != TL or s[8] != "|" or not s[:8].isdigit():
return None
n = int(s[:8])
for i in range(9, TL):
if s[i] != chr(ord("A") + (n * 7 + slot * 3 + i) % 26):
return None
return n
async def main():
res = {"host": HOST}
stopped = {"t": None}
async def on_stop(expected):
stopped["t"] = time.monotonic()
cli = await connect(HOST, on_stop=on_stop)
info = await cli.device_info()
res["device"] = info.name
res["version"] = info.esphome_version
ents, _ = await cli.list_entities_services()
res["entities"] = len(ents)
by_key = {e.key: e for e in ents}
names = {e.object_id: e for e in ents}
flood = names["flood"]
bump = names["bump"]
state = {}
stats = {"states": 0, "bad_sensor": 0, "bad_text": 0, "burst_states": 0}
def on_state(s):
e = by_key.get(s.key)
if e is None:
return
stats["states"] += 1
oid = e.object_id
if isinstance(s, SensorState) and oid.startswith("s") and oid[1:].isdigit():
i = int(oid[1:])
stats["burst_states"] += 1
if not s.missing_state and (
int(s.state) % 100 != i or s.state != int(s.state)
):
stats["bad_sensor"] += 1
if isinstance(s, TextSensorState) and oid.startswith("t") and oid[1:].isdigit():
stats["burst_states"] += 1
if not s.missing_state and txt_ok(s.state, int(oid[1:])) is None:
stats["bad_text"] += 1
state[oid] = s.state
cli.subscribe_states(on_state)
await asyncio.sleep(3)
res["initial_states"] = len(state)
b0 = state.get("bump_count")
b0 = 0 if b0 is None or math.isnan(b0) else int(b0)
for _ in range(5):
cli.button_command(bump.key)
await asyncio.sleep(0.5)
await asyncio.sleep(3)
res["bump_delta"] = int(state.get("bump_count", 0)) - b0
o0 = (
state.get("ovf_enqueue"),
state.get("ovf_drain"),
state.get("ovf_enqueue_bytes"),
state.get("ovf_enqueue_fail"),
)
sock = cli._connection._socket
transport = next(
o
for o in gc.get_objects()
if isinstance(o, asyncio.Transport)
and not o.is_closing()
and o.get_extra_info("socket") is not None
and o.get_extra_info("socket").fileno() == sock.fileno()
)
cli.switch_command(flood.key, True)
await asyncio.sleep(3)
for _ in range(3):
transport.pause_reading()
await asyncio.sleep(6)
transport.resume_reading()
await asyncio.sleep(4)
cli.switch_command(flood.key, False)
await asyncio.sleep(6)
fN = state.get("flood_n")
final_ok = (
fN is not None
and not math.isnan(fN)
and all(state.get(f"s{i}") == fN * 100 + i for i in range(60))
and all(txt_ok(state.get(f"t{j}", ""), j) == int(fN) for j in range(4))
)
o1 = (
state.get("ovf_enqueue"),
state.get("ovf_drain"),
state.get("ovf_enqueue_bytes"),
state.get("ovf_enqueue_fail"),
)
res.update(stats)
res["flood_n"] = fN
res["final_states_match"] = final_ok
# a counter with no state (missing or NaN) gives no delta, which fails the run
for name, a, b in zip(
(
"ovf_enqueue_delta",
"ovf_drain_delta",
"ovf_enqueue_bytes_delta",
"ovf_enqueue_fail_delta",
),
o0,
o1,
strict=True,
):
ok_ab = all(x is not None and not math.isnan(x) for x in (a, b))
res[name] = b - a if ok_ab else None
res["dropped"] = stopped["t"] is not None
await cli.disconnect()
res["pass"] = (
not res["dropped"]
and final_ok
and res["bump_delta"] == 5
and stats["bad_sensor"] == 0
and stats["bad_text"] == 0
and (res["ovf_enqueue_delta"] or 0) > 0
and (res["ovf_drain_delta"] or 0) > 0
# a refused enqueue loses a message
and res["ovf_enqueue_fail_delta"] == 0
)
print(json.dumps(res))
pathlib.Path(OUT).write_text(json.dumps(res, indent=1), encoding="utf-8")
return 0 if res["pass"] else 1
sys.exit(asyncio.run(main()))
@@ -1,67 +0,0 @@
"""gen_sensors.py: write putbe-sensors.yaml for putbe-common.yaml.
It holds the BS_COUNT (60) template sensors bs0..bs59 and the BT_COUNT (4) 800 char text sensors bt0..bt3 that the
flood lambda publishes through the BS and BT pointer arrays of txt.hpp (filled by a boot lambda), and the remote
entity_transport sensors and binary sensors: count, flt and bin from both the base (${peer}b) and the PR (${peer}p)
name of the peer, over both links (et_plain, et_full).
"""
import pathlib
BS_COUNT = 60 # txt.hpp
BT_COUNT = 4 # txt.hpp
LINKS = ("plain", "full")
PEERS = ("b", "p")
def ptr_fill(kind: str, arr: str, prefix: str, n: int) -> list[str]:
ids = ", ".join(f"id({prefix}{i})" for i in range(n))
return [
" - lambda: |-",
f" {kind} *all[] = {{{ids}}};",
f" for (size_t i = 0; i < {arr}_COUNT; i++) {arr}[i] = all[i];",
]
def remote(link: str, peer: str, rid: str, name: str) -> list[str]:
return [
" - platform: entity_transport",
f" entity_transport_id: et_{link}",
f" id: r_{link}_{peer}_{rid}",
f" name: R {link} {peer} {name}",
f" provider: ${{peer}}{peer}",
f" remote_id: {rid}",
]
out = ["esphome:", " on_boot:", " priority: 600", " then:"]
out += ptr_fill("sensor::Sensor", "BS", "bs", BS_COUNT)
out += ptr_fill("text_sensor::TextSensor", "BT", "bt", BT_COUNT)
out.append("sensor:")
for i in range(BS_COUNT):
out += [
" - platform: template",
f" id: bs{i}",
f" name: S{i}",
" update_interval: never",
" accuracy_decimals: 0",
]
for link in LINKS:
for peer in PEERS:
out += remote(link, peer, "count", "count")
out += remote(link, peer, "flt", "flt") + [" accuracy_decimals: 3"]
out.append("binary_sensor:")
for link in LINKS:
for peer in PEERS:
out += remote(link, peer, "bin", "bin")
out.append("text_sensor:")
for i in range(BT_COUNT):
out += [
" - platform: template",
f" id: bt{i}",
f" name: T{i}",
" update_interval: never",
]
pathlib.Path(__file__).with_name("putbe-sensors.yaml").write_text(
"\n".join(out) + "\n", encoding="utf-8"
)
@@ -1,30 +0,0 @@
#pragma once
// Counts calls into the API overflow buffer through -Wl,--wrap (test only).
#include <cstddef>
#include <cstdint>
#include <sys/types.h>
struct iovec;
extern "C" {
uint32_t ovf_enq_calls = 0;
uint32_t ovf_enq_bytes = 0;
uint32_t ovf_drain_calls = 0;
uint32_t ovf_enq_fails = 0; // the buffer refused a message: it is lost
bool __real__ZN7esphome3api17APIOverflowBuffer11enqueue_iovEPK5iovecijj(void *self, const iovec *iov, int n,
size_t total, size_t skip);
bool __wrap__ZN7esphome3api17APIOverflowBuffer11enqueue_iovEPK5iovecijj(void *self, const iovec *iov, int n,
size_t total, size_t skip) {
bool ok = __real__ZN7esphome3api17APIOverflowBuffer11enqueue_iovEPK5iovecijj(self, iov, n, total, skip);
if (ok) {
ovf_enq_calls++;
ovf_enq_bytes += total - skip;
} else {
ovf_enq_fails++;
}
return ok;
}
ssize_t OVF_DRAIN_REAL(void *self, void *sock);
ssize_t OVF_DRAIN_WRAP(void *self, void *sock) {
ovf_drain_calls++;
return OVF_DRAIN_REAL(self, sock);
}
}
@@ -1,156 +0,0 @@
esphome:
name: ${name}
includes:
- ovfwrap.hpp
- txt.hpp
build_flags:
- -Wl,--wrap=_ZN7esphome3api17APIOverflowBuffer11enqueue_iovEPK5iovecijj
- -Wl,--wrap=${drain_sym}
- -DOVF_DRAIN_WRAP=__wrap_${drain_sym}
- -DOVF_DRAIN_REAL=__real_${drain_sym}
logger:
level: DEBUG
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
power_save_mode: none
api:
reboot_timeout: 0s
encryption:
key: !secret api_key
ota:
- platform: esphome
packages:
# the flooded sensors and text sensors, their pointer arrays and the remote entities; run gen_sensors.py once
bs: !include putbe-sensors.yaml
globals:
- id: flood_n
type: uint32_t
initial_value: "0"
- id: bump_n
type: uint32_t
initial_value: "0"
- id: tick
type: uint32_t
initial_value: "0"
switch:
- platform: template
id: flood
name: Flood
optimistic: true
button:
- platform: template
name: Bump
on_press:
- lambda: |-
id(bump_n)++;
ESP_LOGI("bump", "press %u", (unsigned) id(bump_n));
id(bump_sensor).publish_state(id(bump_n));
interval:
- interval: 50ms
then:
- if:
condition:
switch.is_on: flood
then:
- lambda: |-
uint32_t n = ++id(flood_n);
for (size_t i = 0; i < BS_COUNT; i++)
BS[i]->publish_state(n * 100 + i);
for (size_t i = 0; i < BT_COUNT; i++)
BT[i]->publish_state(make_txt(n, i));
id(flood_sensor).publish_state(n);
- interval: 1s
then:
- lambda: |-
uint32_t t = ++id(tick);
id(loc_count).publish_state(t);
id(loc_float).publish_state(12345.678f + t * 0.001f - (t % 7) * 1000.5f);
id(loc_bin).publish_state((t / 3) % 2 == 1);
id(ovf_enq).publish_state(ovf_enq_calls);
id(ovf_enq_b).publish_state(ovf_enq_bytes);
id(ovf_drain).publish_state(ovf_drain_calls);
id(ovf_enq_fail).publish_state(ovf_enq_fails);
udp:
- id: udp_plain
port: 18611
addresses: ["${peer_ip}"]
- id: udp_full
port: 18612
addresses: ["${peer_ip}"]
packet_interface:
- platform: udp
id: pi_plain
udp_id: udp_plain
- platform: udp
id: pi_full
udp_id: udp_full
addressing:
address: ${my_addr}
destination: ${peer_addr}
encryption:
key: putbe test key
rolling_code_enable: true
reliability:
fragmentation:
session:
entity_transport:
- id: et_plain
packet_interface: pi_plain
update_interval: 2s
sensors:
- id: loc_count
broadcast_id: count
- id: loc_float
broadcast_id: flt
binary_sensors:
- id: loc_bin
broadcast_id: bin
- id: et_full
packet_interface: pi_full
sensors:
- id: loc_count
broadcast_id: count
- id: loc_float
broadcast_id: flt
binary_sensors:
- id: loc_bin
broadcast_id: bin
sensor:
- platform: template
id: flood_sensor
name: Flood N
update_interval: never
- platform: template
id: bump_sensor
name: Bump count
update_interval: never
- platform: template
id: loc_count
name: Local count
update_interval: never
- platform: template
id: loc_float
name: Local float
accuracy_decimals: 3
update_interval: never
- platform: template
id: ovf_enq
name: Ovf enqueue
update_interval: never
- platform: template
id: ovf_enq_b
name: Ovf enqueue bytes
update_interval: never
- platform: template
id: ovf_drain
name: Ovf drain
update_interval: never
- platform: template
id: ovf_enq_fail
name: Ovf enqueue fail
update_interval: never
binary_sensor:
- platform: template
id: loc_bin
name: Local bin
@@ -1,19 +0,0 @@
#pragma once
#include <string>
#include <cstdio>
// the flooded template sensors and text sensors, filled at boot by putbe-sensors.yaml (gen_sensors.py)
static constexpr size_t BS_COUNT = 60;
static esphome::sensor::Sensor *BS[BS_COUNT];
static constexpr size_t BT_COUNT = 4;
static esphome::text_sensor::TextSensor *BT[BT_COUNT];
// 800 chars: 8 digit n, '|', then letters from n and the slot
static std::string make_txt(uint32_t n, int slot) {
std::string s(800, ' ');
char head[16];
snprintf(head, sizeof(head), "%08u|", (unsigned) n);
for (int i = 0; i < 9; i++)
s[i] = head[i];
for (int i = 9; i < 800; i++)
s[i] = 'A' + (n * 7 + slot * 3 + i) % 26;
return s;
}
@@ -4,7 +4,6 @@ Import it with this directory on sys.path (the drivers add their parent director
"""
import asyncio
from collections.abc import Awaitable, Callable
import contextlib
import os
import re
@@ -28,32 +27,19 @@ FATAL = (
)
async def connect(
host: str,
tries: int = 30,
timeout: float = 10,
port: int = 6053,
on_stop: Callable[[bool], Awaitable[None]] | None = None,
) -> APIClient:
async def connect(host: str, tries: int = 30, timeout: float = 10) -> APIClient:
"""Connect and log in, retrying once a second; exits after TRIES failures with the last error.
API_KEY in the env selects Noise; unset, empty or - is plaintext. ON_STOP is called when the connection ends.
API_KEY in the env selects Noise; unset, empty or - is plaintext.
"""
key = os.environ.get("API_KEY")
last: BaseException | None = None
for _ in range(tries):
c = APIClient(
host, port, None, noise_psk=None if key in (None, "", "-") else key
host, 6053, None, noise_psk=None if key in (None, "", "-") else key
)
live = [False]
# a failed attempt's cleanup may call on_stop too; pass on only stops of the returned client
async def stop(expected: bool, live: list[bool] = live) -> None:
if live[0] and on_stop is not None:
await on_stop(expected)
try:
await asyncio.wait_for(c.connect(on_stop=stop, login=True), timeout)
await asyncio.wait_for(c.connect(login=True), timeout)
except FATAL as e:
sys.exit(f"no API connection to {host}: {type(e).__name__}: {e}")
except Exception as e: # noqa: BLE001
@@ -63,7 +49,6 @@ async def connect(
await c.disconnect(force=True)
await asyncio.sleep(1)
else:
live[0] = True
return c
sys.exit(
f"no API connection to {host} after {tries} tries: {type(last).__name__}: {last}"
@@ -73,80 +58,3 @@ async def connect(
def log_text(msg) -> str:
"""The message of a subscribe_logs callback as text, without ANSI colour codes."""
return ANSI.sub("", msg.message.decode(errors="replace"))
def stop_event() -> tuple[asyncio.Event, Callable[[bool], Awaitable[None]]]:
"""An Event and an on_stop callback for connect() that sets it when the connection ends."""
ev = asyncio.Event()
async def on_stop(expected: bool) -> None:
ev.set()
return ev, on_stop
async def ble_connect(
cli: APIClient,
addr: int,
on_lost: Callable[[int], None] | None = None,
advert_timeout: float = 60,
timeout: float = 30,
) -> tuple[bool, int, int, Callable[[], None]]:
"""Wait for an advert from ADDR, then connect through the proxy without a cache (like Home Assistant).
Returns (connected, mtu, error, cancel); a missing advert or a connect timeout gives (False, 0, -1, cancel), so
callers can retry. ON_LOST(error) is called when the link drops later.
"""
seen = asyncio.Event()
unsub = cli.subscribe_bluetooth_le_raw_advertisements(
lambda m: (
seen.set() if any(a.address == addr for a in m.advertisements) else None
)
)
try:
await asyncio.wait_for(seen.wait(), advert_timeout)
except TimeoutError:
return False, 0, -1, lambda: None
finally:
unsub()
first: asyncio.Queue = asyncio.Queue()
pending = True
def on_state(connected: bool, mtu: int, err: int) -> None:
nonlocal pending
if pending:
pending = False
first.put_nowait((connected, mtu, err))
elif not connected and on_lost is not None:
on_lost(err)
cancel = await cli.bluetooth_device_connect(
addr,
on_state,
timeout=timeout,
address_type=0,
feature_flags=0xFF,
has_cache=False,
)
try:
connected, mtu, err = await asyncio.wait_for(first.get(), timeout + 5)
except TimeoutError:
return False, 0, -1, cancel
return connected, mtu, err, cancel
async def enable_notify(
cli: APIClient, addr: int, ch, on_notify: Callable[[int, bytearray], None]
) -> bool:
"""Start notify on characteristic CH and write its first CCCD (2902) like Home Assistant.
Returns False when the characteristic lists no CCCD.
"""
await cli.bluetooth_gatt_start_notify(addr, ch.handle, on_notify)
cccd = next(
(d for d in ch.descriptors if d.uuid.lower().startswith("00002902")), None
)
if cccd is None:
return False
await cli.bluetooth_gatt_write_descriptor(addr, cccd.handle, b"\x01\x00")
return True
@@ -1,31 +0,0 @@
#pragma once
// Test only: swaps the first two records of an ATT Read By Type Response whose first record declares characteristic
// 0xFFF1, the way some peripherals answer out of handle order (esphome#22518). Count of swaps in attswap_count.
#include <cstdint>
#include <cstring>
#include "esphome/core/log.h"
struct AttSwapHdr {
uint16_t event, len, offset, layer_specific;
};
extern "C" void __real_l2c_rcv_acl_data(void *p_msg);
static volatile uint32_t attswap_count = 0;
extern "C" void __wrap_l2c_rcv_acl_data(void *p_msg) {
auto *h = static_cast<AttSwapHdr *>(p_msg);
uint8_t *p = reinterpret_cast<uint8_t *>(h + 1) + h->offset;
// HCI ACL header (4), L2CAP header (4), then ATT: opcode, record length, records
if (h->len >= 4 + 4 + 2 + 14) {
uint16_t cid = p[6] | (p[7] << 8);
uint8_t *att = p + 8;
// record: handle(2) properties(1) value handle(2) uuid16(2)
if (cid == 0x0004 && att[0] == 0x09 && att[1] == 7 && att[2 + 5] == 0xF1 && att[2 + 6] == 0xFF) {
uint8_t tmp[7];
memcpy(tmp, att + 2, 7);
memcpy(att + 2, att + 9, 7);
memcpy(att + 9, tmp, 7);
attswap_count = attswap_count + 1;
}
}
__real_l2c_rcv_acl_data(p_msg);
}
@@ -1,11 +0,0 @@
"""Test only: wrap Bluedroid's l2c_rcv_acl_data so attswap.h can reorder a Read By Type response."""
import esphome.codegen as cg
import esphome.config_validation as cv
from esphome.types import ConfigType
CONFIG_SCHEMA = cv.Schema({})
async def to_code(config: ConfigType) -> None:
cg.add_build_flag("-Wl,--wrap=l2c_rcv_acl_data")
@@ -1,5 +0,0 @@
substitutions:
name: attprx6
idf: 5.5.6
packages:
c: !include proxy.yaml
@@ -1,125 +0,0 @@
"""Through the proxy (no cache): print the full GATT table, read FFF1/FFF2/FFF3 by handle, and enable notify on
FFF1, FFF2 and 2ACD (writing the CCCD the way Home Assistant does), counting what arrives per handle for 6 s.
usage: gattcheck.py PROXY PEER_MAC env: API_KEY (the proxy's api encryption key)
Exits 1 when a characteristic or CCCD is missing, a read or notification carries another characteristic's value,
a notify fails, or a notifying characteristic sends nothing."""
import asyncio
import os
import sys
from aioesphomeapi import APIClient
SFX = "-0000-1000-8000-00805f9b34fb"
def short(u: str) -> str:
return u[4:8] if u.endswith(SFX) else u
async def main(host: str, peer: str) -> None:
addr = int(peer.replace(":", ""), 16)
cli = APIClient(host, 6053, None, noise_psk=os.environ["API_KEY"])
await cli.connect(login=True)
seen = asyncio.Event()
cli.subscribe_bluetooth_le_raw_advertisements(
lambda m: (
seen.set() if any(a.address == addr for a in m.advertisements) else None
)
)
await asyncio.wait_for(seen.wait(), 30)
state: asyncio.Queue = asyncio.Queue()
cancel = await cli.bluetooth_device_connect(
addr,
lambda c, mtu, err: state.put_nowait((c, mtu, err)),
timeout=20,
address_type=0,
feature_flags=0xFF,
has_cache=False,
)
c, mtu, err = await asyncio.wait_for(state.get(), 25)
if not c:
sys.exit(f"connect failed err={err}")
svcs = await cli.bluetooth_gatt_get_services(addr)
chars = {}
for s in svcs.services:
print(f"SVC {short(s.uuid)} 0x{s.handle:04X}", flush=True)
for ch in s.characteristics:
chars[short(ch.uuid)] = ch
print(
f" CHR {short(ch.uuid)} 0x{ch.handle:04X} props=0x{ch.properties:02X}",
flush=True,
)
for d in ch.descriptors:
print(f" DSC {short(d.uuid)} 0x{d.handle:04X}", flush=True)
wanted = ("fff1", "fff2", "fff3", "2acc", "2acd")
problems = [f"{u} missing" for u in wanted if u not in chars]
for u in ("fff1", "fff2", "fff3", "2acc"):
if u in chars:
v = bytes(await cli.bluetooth_gatt_read(addr, chars[u].handle))
text = v.decode(errors="replace")
print(f"READ {u} 0x{chars[u].handle:04X} -> {text}", flush=True)
# every server value starts with its own UUID, so a mixed-up handle shows here
if not text.startswith(f"{u}-"):
problems.append(f"{u} read returned {text!r}")
got: dict[int, list[str]] = {}
def on_notify(handle: int, data: bytearray) -> None:
got.setdefault(handle, []).append(bytes(data).decode(errors="replace"))
for u in ("fff1", "fff2", "2acd"):
if u in chars:
try:
await cli.bluetooth_gatt_start_notify(addr, chars[u].handle, on_notify)
# like Home Assistant's client: write the first CCCD listed under the characteristic
cccd = next(
(d for d in chars[u].descriptors if short(d.uuid) == "2902"), None
)
if cccd is None:
print(
f"NOTIFY {u} 0x{chars[u].handle:04X} NO CCCD in its descriptor list",
flush=True,
)
problems.append(f"{u} has no CCCD")
continue
await cli.bluetooth_gatt_write_descriptor(
addr, cccd.handle, b"\x01\x00"
)
print(
f"NOTIFY {u} 0x{chars[u].handle:04X} enabled, CCCD 0x{cccd.handle:04X} written",
flush=True,
)
except Exception as exc: # noqa: BLE001
print(
f"NOTIFY {u} 0x{chars[u].handle:04X} FAILED {type(exc).__name__}: {exc}",
flush=True,
)
problems.append(f"{u} notify failed")
await asyncio.sleep(6)
if not got:
print("GOT nothing", flush=True)
for h, vals in sorted(got.items()):
name = next((u for u, ch in chars.items() if ch.handle == h), "?")
print(
f"GOT 0x{h:04X} ({name}) x{len(vals)}: {vals[0]} .. {vals[-1]}", flush=True
)
problems.extend(
f"0x{h:04X} ({name}) notified {v!r}"
for v in vals
if name == "?" or not v.startswith(f"{name}-")
)
# the server notifies FFF1 and 2ACD every second
problems.extend(
f"{u} sent nothing"
for u in ("fff1", "2acd")
if u in chars and chars[u].handle not in got
)
await cli.bluetooth_device_disconnect(addr)
cancel()
await cli.disconnect()
if problems:
sys.exit("FAIL: " + ", ".join(problems))
print("PASS", flush=True)
asyncio.run(main(sys.argv[1], sys.argv[2]))
@@ -1,56 +0,0 @@
"""Connect to PEER_MAC through the proxy like Home Assistant (v3 without cache) and print the services it returns.
usage: gattget.py PROXY PEER_MAC ROUNDS env: API_KEY (the proxy's api encryption key)"""
import asyncio
import os
import sys
from aioesphomeapi import APIClient
async def main(host: str, peer: str, rounds: int) -> None:
addr = int(peer.replace(":", ""), 16)
cli = APIClient(host, 6053, None, noise_psk=os.environ["API_KEY"])
await cli.connect(login=True)
seen = asyncio.Event()
def adv(msg) -> None:
if any(a.address == addr for a in msg.advertisements):
seen.set()
cli.subscribe_bluetooth_le_raw_advertisements(adv)
await asyncio.wait_for(seen.wait(), 30)
failed = 0
for r in range(1, rounds + 1):
state: asyncio.Queue = asyncio.Queue()
cancel = await cli.bluetooth_device_connect(
addr,
lambda c, mtu, err, q=state: q.put_nowait((c, mtu, err)),
timeout=20,
address_type=0,
feature_flags=0xFF,
has_cache=False,
)
c, mtu, err = await asyncio.wait_for(state.get(), 25)
if not c:
print(f"R{r} connect failed err={err}", flush=True)
failed += 1
cancel()
await asyncio.sleep(5)
continue
svcs = await cli.bluetooth_gatt_get_services(addr)
sfx = "-0000-1000-8000-00805f9b34fb"
out = [
f"{s.uuid[4:8] if s.uuid.endswith(sfx) else s.uuid}({len(s.characteristics)})"
for s in svcs.services
]
print(f"R{r} mtu={mtu} services: {' '.join(out)}", flush=True)
await cli.bluetooth_device_disconnect(addr)
cancel()
await asyncio.sleep(5)
await cli.disconnect()
if failed:
sys.exit(f"{failed} of {rounds} rounds did not connect")
asyncio.run(main(sys.argv[1], sys.argv[2], int(sys.argv[3])))
@@ -1,37 +0,0 @@
# Proxy (classic ESP32): bluetooth_proxy without service cache, encrypted API, the reorder wrapper.
# Substitutions: name, idf.
# Keep the per-build file in this directory: attswap.hpp and components/ are found relative to it.
esphome:
name: ${name}
includes: [attswap.hpp]
esp32:
board: esp32dev
framework:
type: esp-idf
version: ${idf}
logger:
level: DEBUG
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
api:
reboot_timeout: 0s
# a plaintext proxy can be taken over by any Home Assistant on the network mid-test
encryption:
key: !secret api_key
ota:
- platform: esphome
external_components:
- source: {type: local, path: components}
components: [attswap]
attswap:
esp32_ble_tracker:
scan_parameters:
active: true
bluetooth_proxy:
active: true
cache_services: false
interval:
- interval: 5s
then:
- lambda: 'ESP_LOGI("swap", "attswap swaps=%u", (unsigned) attswap_count);'
@@ -1,36 +0,0 @@
#!/bin/bash
# run.sh PROXY_YAML [ROUNDS]: build server.yaml and PROXY_YAML from TREE, flash both, capture the proxy and read the
# server's services through it ROUNDS times (default 3). Prints the services per round and the Bluedroid discovery
# errors. Exit 0: every round got all services; 1: a round got a partial table (the bug); 2: setup failed, a round
# did not connect, or the swap never fired (the result would be meaningless). PROXY_YAML must sit in this
# directory, next to attswap.hpp and components/.
# env: TREE, SERVER_YAML (default server.yaml), API_KEY (proxy api key), SERVER_PORT, PROXY_PORT, SERVER_BLE_MAC
# (the server's BLE address), SC (path to serial_capture.py), PY
D=$(cd "$(dirname "$0")" && pwd); Y=$(cd "$(dirname "${1:?proxy yaml}")" && pwd)/$(basename "$1"); R=${2:-3}
W=${TREE:?set TREE}; SP=${SERVER_PORT:?set SERVER_PORT}; PP=${PROXY_PORT:?set PROXY_PORT}; MAC=${SERVER_BLE_MAC:?set SERVER_BLE_MAC}
API_KEY=${API_KEY:?set API_KEY}; SC=${SC:?set SC}; PY=${PY:-$W/venv/bin/python}; N=$(basename "$Y" .yaml); O=$D/out-$N; mkdir -p "$O"
(cd "$W" && PYTHONPATH=$W $PY -m esphome -s name attsrv run --no-logs "$D/${SERVER_YAML:-server.yaml}" --device "$SP") > "$O/server.log" 2>&1 ||
{ echo "server build or upload failed"; exit 2; }
(cd "$W" && PYTHONPATH=$W $PY -m esphome run --no-logs "$Y" --device "$PP") > "$O/proxy-upload.log" 2>&1 ||
{ echo "$N build or upload failed"; exit 2; }
rm -f "$O/proxy.txt"
$PY "$SC" "$PP" $((R * 30 + 160)) "$O/proxy.txt" --reset > "$O/capture.log" 2>&1 &
CAP=$!
IP=""
for _ in $(seq 40); do
IP=$(LC_ALL=C grep -a -o -E "IP Address: [0-9.]+" "$O/proxy.txt" | tail -1 | cut -d' ' -f3)
[ -n "$IP" ] && break; sleep 2
done
[ -n "$IP" ] || { kill "$CAP"; echo "$N no IP"; exit 2; }
sleep 5
timeout $((R * 30 + 60)) $PY "$D/gattget.py" "$IP" "$MAC" "$R" | tee "$O/client.txt"
[ "${PIPESTATUS[0]}" = 0 ] || { kill "$CAP"; echo "$N client failed"; exit 2; }
wait "$CAP" || { echo "$N capture failed (see $O/capture.log)"; exit 2; }
echo "== $N"
LC_ALL=C grep -a -E "invalid handle|Service discovery complete|attswap swaps=[1-9]" "$O/proxy.txt" | sed 's/^\[[0-9:]*\]//' |
sort | uniq -c
grep -a -q -E "attswap swaps=[1-9]" "$O/proxy.txt" || { echo "$N INVALID: the reorder never fired"; exit 2; }
# every round must list the reordered service (fff0) and the one after it (1826) with all their characteristics
grep -a -q "^R[0-9]* mtu=" "$O/client.txt" || { echo "$N no client output"; exit 2; }
grep -a "^R[0-9]* mtu=" "$O/client.txt" | grep -v -q "fff0(6) 1826(2)" && exit 1
exit 0
@@ -1,45 +0,0 @@
# Peripheral (ESP32-S3) for gattcheck.py: FFF1 to FFF6 each hold their own value, FFF1, FFF2 and 2ACD can notify,
# and FFF1 and 2ACD send a changing value every second. Substitution: name.
esphome:
name: ${name}
esp32:
variant: esp32s3
framework:
type: esp-idf
logger:
hardware_uart: USB_SERIAL_JTAG
esp32_ble:
name: ${name}
esp32_ble_server:
manufacturer: ESPHome
model: attcheck
services:
- uuid: FFF0
advertise: true
characteristics:
- {id: c1, uuid: FFF1, read: true, notify: true, value: "fff1-v"}
- {uuid: FFF2, read: true, notify: true, value: "fff2-v"}
- {uuid: FFF3, read: true, value: "fff3-v"}
- {uuid: FFF4, read: true, value: "fff4-v"}
- {uuid: FFF5, read: true, value: "fff5-v"}
- {uuid: FFF6, read: true, value: "fff6-v"}
- uuid: "1826"
characteristics:
- {uuid: 2ACC, read: true, value: "2acc-v"}
- {id: cd, uuid: 2ACD, read: true, notify: true, value: "2acd-0"}
on_connect:
- logger.log: {format: "SRV connected", level: INFO}
globals:
- {id: n, type: int, initial_value: "0"}
interval:
- interval: 1s
then:
- lambda: id(n)++;
- ble_server.characteristic.set_value:
id: cd
value: !lambda 'std::string v = "2acd-" + to_string(id(n)); return std::vector<uint8_t>(v.begin(), v.end());'
- ble_server.characteristic.notify: {id: cd}
- ble_server.characteristic.set_value:
id: c1
value: !lambda 'std::string v = "fff1-n" + to_string(id(n)); return std::vector<uint8_t>(v.begin(), v.end());'
- ble_server.characteristic.notify: {id: c1}
@@ -1,32 +0,0 @@
# Peripheral (ESP32-S3): a 6-characteristic service 0xFFF0, then an FTMS-like 0x1826 service. Substitution: name.
esphome:
name: ${name}
esp32:
variant: esp32s3
framework:
type: esp-idf
logger:
hardware_uart: USB_SERIAL_JTAG
esp32_ble:
name: ${name}
esp32_ble_server:
manufacturer: ESPHome
model: g22518
services:
- uuid: FFF0
advertise: true
characteristics:
- {uuid: FFF1, read: true, value: [1]}
- {uuid: FFF2, read: true, value: [2]}
- {uuid: FFF3, read: true, value: [3]}
- {uuid: FFF4, read: true, value: [4]}
- {uuid: FFF5, read: true, value: [5]}
- {uuid: FFF6, read: true, value: [6]}
- uuid: "1826"
characteristics:
- {uuid: 2ACC, read: true, value: [0, 0, 0, 0]}
- {uuid: 2ACD, read: true, notify: true, value: [0, 0]}
on_connect:
- logger.log: {format: "SRV connected", level: INFO}
on_disconnect:
- logger.log: {format: "SRV disconnected", level: INFO}
@@ -1,48 +0,0 @@
# Joins the DUT's open AP, fetches the portal page every 5 s and logs JOIN and STAT lines. Substitutions: name, board,
# logger_uart, ap_ssid (same as dut.yaml).
substitutions:
board: esp32-c3-devkitm-1
logger_uart: USB_SERIAL_JTAG
ap_ssid: bleaptest
esphome:
name: ${name}
esp32:
board: ${board}
framework:
type: esp-idf
logger:
hardware_uart: ${logger_uart}
wifi:
ssid: ${ap_ssid}
reboot_timeout: 0s
on_connect:
- lambda: |-
id(joins)++;
ESP_LOGI("cli", "JOIN %d at %u ms", id(joins), (unsigned) millis());
on_disconnect:
- lambda: ESP_LOGI("cli", "LEFT at %u ms", (unsigned) millis());
http_request:
timeout: 5s
globals:
- {id: joins, type: int, initial_value: "0"}
- {id: pages_ok, type: int, initial_value: "0"}
- {id: pages_fail, type: int, initial_value: "0"}
interval:
- interval: 5s
then:
- if:
condition: wifi.connected
then:
- http_request.get:
url: http://192.168.4.1/
on_response:
then:
- lambda: if (response->status_code == 200) id(pages_ok)++; else id(pages_fail)++;
on_error:
then:
- lambda: id(pages_fail)++;
- interval: 15s
then:
- lambda: |-
ESP_LOGI("cli", "STAT up=%u joins=%d connected=%d pages_ok=%d pages_fail=%d", (unsigned) (millis() / 1000),
id(joins), (int) wifi::global_wifi_component->is_connected(), id(pages_ok), id(pages_fail));
@@ -1,23 +0,0 @@
# Soft AP only (no station network) with captive_portal and bluetooth_proxy, as in #22534. Substitutions: name,
# board, logger_uart (USB_SERIAL_JTAG on native USB chips, UART0 on a classic devkit), ap_ssid.
substitutions:
board: esp32dev
logger_uart: UART0
ap_ssid: bleaptest
esphome:
name: ${name}
esp32:
board: ${board}
framework:
type: esp-idf
logger:
hardware_uart: ${logger_uart}
api:
reboot_timeout: 0s
ota:
- platform: esphome
wifi:
ap:
ssid: ${ap_ssid}
captive_portal:
bluetooth_proxy:
@@ -1,29 +0,0 @@
#!/bin/bash
# run.sh DUT_YAML SECONDS: erase and flash DUT_YAML from TREE, let it boot, then reset the client (already flashed with
# client.yaml) and capture both for SECONDS. Prints the generated scan window, the client's first JOIN and last STAT
# lines, its disconnect reasons and the DUT's resets. One AP with this SSID at a time: ports in PARK are held in their
# ROM bootloader first. The yaml file name must equal its `name:` (the build directory is found by it).
# Exit 0: pass; 1: the run finished and failed the checks; 2: setup or capture failed.
# env: TREE, DUT_PORT, CLIENT_PORT, PARK (space separated ports, optional), SC (path to serial_capture.py), PY
Y=$(cd "$(dirname "${1:?dut yaml}")" && pwd)/$(basename "$1"); T=${2:?seconds}; N=$(basename "$Y" .yaml)
W=${TREE:?set TREE}; DUT=${DUT_PORT:?set DUT_PORT}; CLI=${CLIENT_PORT:?set CLIENT_PORT}; SC=${SC:?set SC}
PY=${PY:-$W/venv/bin/python}; O=$(dirname "$Y")/out-$N; mkdir -p "$O"
for p in $PARK; do $PY -m esptool --port "$p" --after no-reset chip-id > /dev/null 2>&1 || { echo "park $p failed"; exit 2; }; done
(cd "$W" && PYTHONPATH=$W $PY -m esphome compile "$Y") > "$O/compile.log" 2>&1 || { echo "$N compile failed"; exit 2; }
$PY -m esptool --port "$DUT" erase_flash > "$O/erase.log" 2>&1 || { echo "$N erase failed"; exit 2; }
(cd "$W" && PYTHONPATH=$W $PY -m esphome upload "$Y" --device "$DUT") > "$O/upload.log" 2>&1 || { echo "$N upload failed"; exit 2; }
$PY "$SC" "$DUT" $((T + 40)) "$O/dut.txt" --reset > "$O/dut-capture.log" 2>&1 &
CAP=$!
trap 'kill $CAP 2>/dev/null' EXIT
sleep 30
$PY "$SC" "$CLI" "$T" "$O/cli.txt" --reset > "$O/cli-capture.log" 2>&1 || { echo "$N client capture failed"; exit 2; }
wait "$CAP" || { echo "$N DUT capture failed"; exit 2; }
B=$(dirname "$Y")/.esphome/build/$N/src/main.cpp
[ -f "$B" ] || { echo "$N: $B not found (file name differs from name:?)"; exit 2; }
echo "== $N window=$(grep -o 'set_scan_window([0-9]*)' "$B" || echo none)"
J=$(grep -a -m1 "JOIN 1" "$O/cli.txt" | cut -c1-100); echo "${J:-$N NO JOIN}"
STAT=$(grep -a "STAT" "$O/cli.txt" | tail -1 | cut -c1-140); echo "$STAT"
echo "client disconnect reasons: $(grep -a -o "reason='[^']*'" "$O/cli.txt" | sort | uniq -c | tr '\n' ' ')"
echo "dut resets: $(grep -a -c 'rst:' "$O/dut.txt")"
# pass: the client joined and fetched pages, and the DUT reset exactly once (the capture's own reset)
[ -n "$J" ] && [ -n "$STAT" ] && [[ "$STAT" != *"pages_ok=0 "* ]] && [ "$(grep -a -c 'rst:' "$O/dut.txt")" = 1 ]
@@ -1,57 +0,0 @@
"""Count rounds, BTU_TASK crashes, reconnects without reboot and MTU 517 in a central capture.
analyze.py CAPTURE
A round counts from its "start" line (a crashing round never logs "fired"); rounds that log "not queued" or
"aborted" are taken back out and listed apart. Exits 1 when no round counts, since such a capture tested nothing.
"""
import pathlib
import re
import sys
lines = (
pathlib.Path(sys.argv[1]).read_text(encoding="utf-8", errors="replace").splitlines()
)
rounds = crashes = other_panics = boots = connects = mtu517 = mtu_other = 0
not_queued = aborted = skipped = 0
reconnect_ok = reconnect_reboot = 0
pending = False # a round fired and we wait for the next connect
for ln in lines:
if "Stack canary watchpoint triggered (BTU_TASK)" in ln:
crashes += 1
elif "Guru Meditation" in ln or "abort() was called" in ln:
other_panics += 1
if "rst:0x" in ln:
boots += 1
if pending:
reconnect_reboot += 1
pending = False
if "CLI connected" in ln:
connects += 1
if pending:
reconnect_ok += 1
pending = False
if re.search(r"ROUND \d+ start", ln):
rounds += 1
pending = True
if re.search(r"ROUND \d+ (not queued|aborted)", ln):
rounds -= 1
pending = False
not_queued += "not queued" in ln
aborted += "aborted" in ln
skipped += "ROUND skipped" in ln
if m := re.search(r"cfg_mtu status 0, mtu (\d+)", ln):
if m.group(1) == "517":
mtu517 += 1
else:
mtu_other += 1
print(
f"rounds={rounds} btu_canary_crashes={crashes} other_panics={other_panics} boots={boots}"
)
print(
f"connects={connects} reconnect_without_reboot={reconnect_ok} reconnect_after_reboot={reconnect_reboot}"
)
print(f"mtu517={mtu517} mtu_other={mtu_other}")
print(f"not_queued={not_queued} aborted={aborted} skipped={skipped}")
if rounds <= 0:
sys.exit("FAIL: no round reached the BTU queue")
@@ -1,75 +0,0 @@
# Central for the Bluedroid queued MTU recursion repro (esphome#17954). Every connection: wait 1.5 s, then
# suspend BTU_TASK, local disconnect, 4 reads + 1 MTU request (or execute write with exec_write: "true") in one
# batch, resume. ble_client reconnects.
esp32:
board: esp32dev
framework:
type: esp-idf
sdkconfig_options:
CONFIG_FREERTOS_WATCHPOINT_END_OF_STACK: y
logger:
level: DEBUG
esp32_ble_tracker:
scan_parameters:
active: false
globals:
- {id: rounds, type: int, initial_value: "0"}
- {id: chr_handle, type: uint16_t, initial_value: "0"}
ble_client:
- mac_address: ${peer_mac}
id: srv
on_connect:
- logger.log: {format: "CLI connected", level: INFO}
# Runs inside SEARCH_CMPL, before ble_client releases the GATT cache (it has no other nodes)
- lambda: |-
esp_bt_uuid_t uuid = esp32_ble_tracker::ESPBTUUID::from_raw("5ec10001-0000-1000-8000-00805f9b34fb").get_uuid();
esp_gattc_char_elem_t elem;
uint16_t count = 1;
id(chr_handle) = 0;
if (esp_ble_gattc_get_char_by_uuid(id(srv)->get_gattc_if(), id(srv)->get_conn_id(), 1, 0xFFFF, uuid, &elem,
&count) == ESP_GATT_OK && count > 0)
id(chr_handle) = elem.char_handle;
- delay: 1500ms
- lambda: |-
if (!id(srv)->connected()) return;
esp_gatt_if_t gattc_if = id(srv)->get_gattc_if();
uint16_t conn_id = id(srv)->get_conn_id();
uint16_t handle = id(chr_handle);
if (handle == 0) {
ESP_LOGW("btu", "ROUND skipped, no characteristic");
return;
}
int n = ++id(rounds);
ESP_LOGI("btu", "ROUND %d start conn_id=%u handle=0x%04x", n, conn_id, handle);
TaskHandle_t btu = xTaskGetHandle("BTU_TASK");
if (btu == nullptr) {
// vTaskSuspend(nullptr) would suspend this task forever
ESP_LOGE("btu", "ROUND %d aborted, no BTU_TASK", n);
return;
}
vTaskSuspend(btu);
// a rejected disconnect leaves the link up, so the queued calls never take the failing send path
esp_err_t err = esp_ble_gap_disconnect(id(srv)->get_remote_bda());
for (int i = 0; i < 4; i++) {
esp_err_t e = esp_ble_gattc_read_char(gattc_if, conn_id, handle, ESP_GATT_AUTH_REQ_NONE);
if (e != ESP_OK)
err = e;
}
esp_err_t e = ${exec_write} ? esp_ble_gattc_execute_write(gattc_if, conn_id, true)
: esp_ble_gattc_send_mtu_req(gattc_if, conn_id);
if (e != ESP_OK)
err = e;
delay(30);
vTaskResume(btu);
// a round whose calls were rejected never reached the BTU queue, so it tests nothing
if (err != ESP_OK) {
ESP_LOGW("btu", "ROUND %d not queued: %s", n, esp_err_to_name(err));
} else {
ESP_LOGI("btu", "ROUND %d fired", n);
}
on_disconnect:
- logger.log: {format: "CLI disconnected", level: INFO}
@@ -1,9 +0,0 @@
esphome:
name: btuclibase # one name per build (btuclibase, btuclipr)
substitutions:
peer_mac: "AA:BB:CC:DD:EE:FF" # the peer S3 Bluetooth MAC (Wi-Fi MAC + 2)
exec_write: "false"
packages:
common: !include btu-common.yaml
@@ -1,31 +0,0 @@
# Peer for the Bluedroid queued MTU recursion repro: a GATT server with one readable and writable characteristic
esphome:
name: btusrv
esp32:
variant: esp32s3
board: esp32-s3-devkitc-1
framework:
type: esp-idf
logger:
level: DEBUG
hardware_uart: USB_SERIAL_JTAG
esp32_ble:
name: btusrv
esp32_ble_server:
on_connect:
- logger.log: {format: "SRV connected", level: INFO}
on_disconnect:
- logger.log: {format: "SRV disconnected", level: INFO}
services:
- uuid: 5ec10000-0000-1000-8000-00805f9b34fb
advertise: true
characteristics:
- id: numc
uuid: 5ec10001-0000-1000-8000-00805f9b34fb
read: true
write: true
value: [42]
@@ -1,15 +0,0 @@
esphome:
name: btuxwbase # one name per build (btuxwbase, btuxwpr)
substitutions:
peer_mac: "AA:BB:CC:DD:EE:FF" # the peer S3 Bluetooth MAC (Wi-Fi MAC + 2)
exec_write: "true"
esp32:
framework:
sdkconfig_options:
# GATT API trace prints one "GATTC_ExecuteWrite conn_id=" line per execute send
CONFIG_BT_LOG_GATT_TRACE_LEVEL_API: y
packages:
common: !include btu-common.yaml
@@ -1,194 +0,0 @@
#pragma once
// BMP390 bench helpers: register dump, INT data-ready edge counter, start write fault injection (software NACK or
// SDA held through the GPIO matrix) and a bus probe. ESP32-S3, ESP-IDF.
#include <cmath>
#include "esphome/components/i2c/i2c_bus.h"
#include "esphome/core/log.h"
#include "esphome/core/hal.h"
#include "driver/gpio.h"
#include "driver/i2c_master.h"
#include "esp_rom_gpio.h"
#include "soc/gpio_sig_map.h"
#include "esp_heap_caps.h"
#include "soc/gpio_reg.h"
#include "soc/soc.h"
#include "soc/io_mux_reg.h"
#include "soc/gpio_periph.h"
namespace bmpt {
static const char *const TAG = "bmpt";
inline volatile uint32_t g_int_edges = 0;
inline void int_isr(void *) { g_int_edges = g_int_edges + 1; }
inline void int_counter_begin(int pin) {
gpio_config_t c{};
c.pin_bit_mask = 1ULL << pin;
c.mode = GPIO_MODE_INPUT;
c.pull_down_en = GPIO_PULLDOWN_ENABLE;
c.intr_type = GPIO_INTR_POSEDGE;
gpio_config(&c);
esp_err_t e = gpio_install_isr_service(0);
if (e != ESP_OK && e != ESP_ERR_INVALID_STATE)
ESP_LOGE(TAG, "isr service %d", e);
gpio_isr_handler_add((gpio_num_t) pin, int_isr, nullptr);
}
inline int g_sda = -1;
inline int g_port = 0;
inline bool g_held = false;
inline int g_hold_level = 0; // 0: SDA shorted low, 1: SDA disconnected from the controller (open line, pulled high)
inline bool g_bus_reset_on_probe = false;
inline uint32_t g_out_sel = 0;
inline bool g_saved = false;
inline bool g_log_pad_once = true;
inline uint32_t out_sel_reg() { return REG_READ(GPIO_FUNC0_OUT_SEL_CFG_REG + 4 * g_sda); }
inline uint32_t pin_reg() { return REG_READ(GPIO_PIN0_REG + 4 * g_sda); }
inline uint32_t mux_reg() { return REG_READ(GPIO_PIN_MUX_REG[g_sda]); }
inline uint32_t en_bit() { return (REG_READ(GPIO_ENABLE_REG) >> g_sda) & 1; }
inline uint32_t g_pin = 0, g_mux = 0, g_en = 0;
inline void log_pad(const char *when) {
ESP_LOGI(TAG, "PAD %s out_sel=0x%08X pin=0x%08X mux=0x%08X en=%u", when, (unsigned) out_sel_reg(),
(unsigned) pin_reg(), (unsigned) mux_reg(), (unsigned) en_bit());
}
inline void sda_hold() {
if (!g_saved) {
g_out_sel = out_sel_reg(); // routing the i2c driver set up; restored verbatim on release
g_pin = pin_reg();
g_mux = mux_reg();
g_en = en_bit();
log_pad("before-hold");
g_saved = true;
ESP_LOGI(TAG, "SDA out_sel saved 0x%08X (I2CEXT0_SDA_OUT_IDX=%d port=%d)", (unsigned) g_out_sel,
I2CEXT0_SDA_OUT_IDX, g_port);
}
gpio_set_level((gpio_num_t) g_sda, g_hold_level);
esp_rom_gpio_connect_out_signal(g_sda, SIG_GPIO_OUT_IDX, false, false);
g_held = true;
if (g_log_pad_once) {
log_pad("held");
}
}
inline void sda_release() {
gpio_set_level((gpio_num_t) g_sda, 1);
REG_WRITE(GPIO_FUNC0_OUT_SEL_CFG_REG + 4 * g_sda, g_out_sel);
REG_WRITE(GPIO_PIN0_REG + 4 * g_sda, g_pin);
REG_WRITE(GPIO_PIN_MUX_REG[g_sda], g_mux);
if (g_en)
REG_WRITE(GPIO_ENABLE_W1TS_REG, 1u << g_sda);
else
REG_WRITE(GPIO_ENABLE_W1TC_REG, 1u << g_sda);
g_held = false;
if (g_log_pad_once) {
log_pad("released");
g_log_pad_once = false;
}
}
// Wraps the sensor's bus: injects the start write faults and logs every failed transfer
class FaultBus : public esphome::i2c::I2CBus {
public:
explicit FaultBus(esphome::i2c::I2CBus *real) : real_(real) {}
uint32_t hold_until{0}; // millis() deadline of a timed hold
bool arm_start{false}; // hold SDA only during the next forced mode start write (PWR_CTRL mode bits 01)
// until this deadline every transfer fails on its own: free-bus dummy read, hold, transfer, release
uint32_t window_until{0};
bool soft_arm{false}; // next start write fails with NACK in software, the wire is not touched
uint32_t soft_until{0}; // every start write fails with NACK in software until this deadline
uint32_t xfers{0}, fails{0}, start_writes{0}, start_fails{0}, normal_writes{0};
esphome::i2c::ErrorCode write_readv(uint8_t address, const uint8_t *w, size_t wn, uint8_t *r, size_t rn) override {
// mode bits of a PWR_CTRL write: 1 forced (a start), 3 normal
int mode = wn == 2 && w[0] == 0x1B ? (w[1] >> 4) & 3 : -1;
bool is_start = mode == 1;
bool targeted = is_start && this->arm_start;
if (is_start && (this->soft_arm || (int32_t) (this->soft_until - esphome::millis()) > 0)) {
this->soft_arm = false;
this->start_writes++;
this->start_fails++;
this->xfers++;
this->fails++;
ESP_LOGI(TAG, "SOFTFAIL start write reg=0x1B val=0x%02X -> NACK", w[1]);
return esphome::i2c::ERROR_NOT_ACKNOWLEDGED;
}
bool timed = (int32_t) (this->hold_until - esphome::millis()) > 0;
bool window = (int32_t) (this->window_until - esphome::millis()) > 0;
if (window) {
// let the driver run its post-error FSM reset with free lines, so each injected failure starts from a clean bus
uint8_t reg = 0x00, v;
this->real_->write_readv(address, &reg, 1, &v, 1);
targeted = true;
}
if (timed || targeted)
sda_hold(); // reasserted per transfer in case the driver re-routes the pin after an error
if (is_start)
this->start_writes++;
if (mode == 3)
this->normal_writes++;
auto err = this->real_->write_readv(address, w, wn, r, rn);
if (targeted) {
this->arm_start = false;
sda_release();
} else if (!timed && g_held) {
sda_release();
}
this->xfers++;
if (err != esphome::i2c::ERROR_OK) {
this->fails++;
if (is_start)
this->start_fails++;
ESP_LOGI(TAG, "XFERFAIL addr=0x%02X reg=0x%02X wn=%u rn=%u val=0x%02X err=%d start=%d", address,
wn > 0 ? w[0] : 0xFF, (unsigned) wn, (unsigned) rn, wn > 1 ? w[1] : 0, (int) err, is_start);
}
return err;
}
protected:
esphome::i2c::I2CBus *real_;
};
inline FaultBus *g_fb = nullptr;
inline esphome::i2c::I2CBus *g_real = nullptr;
// Release a timed hold once its deadline passes even when no transfer happens.
inline void fault_tick() {
if (g_fb != nullptr && g_held && !g_fb->arm_start && (int32_t) (g_fb->hold_until - esphome::millis()) <= 0)
sda_release();
}
inline bool rd(uint8_t addr, uint8_t reg, uint8_t *v, size_t n = 1) {
return g_real->write_readv(addr, &reg, 1, v, n) == esphome::i2c::ERROR_OK;
}
inline bool wr(uint8_t addr, uint8_t reg, uint8_t v) {
uint8_t b[2] = {reg, v};
return g_real->write_readv(addr, b, 2, nullptr, 0) == esphome::i2c::ERROR_OK;
}
inline void dump_regs(uint8_t addr) {
uint8_t chip = 0, err = 0, ictl = 0, pwr = 0, osr = 0, odr = 0, cfg = 0;
bool ok = rd(addr, 0x00, &chip) && rd(addr, 0x02, &err) && rd(addr, 0x19, &ictl) && rd(addr, 0x1B, &pwr) &&
rd(addr, 0x1C, &osr) && rd(addr, 0x1D, &odr) && rd(addr, 0x1F, &cfg);
ESP_LOGI(TAG, "REGS ok=%d chip=0x%02X err=0x%02X int_ctrl=0x%02X pwr=0x%02X mode=%u osr=0x%02X odr=0x%02X cfg=0x%02X",
ok, chip, err, ictl, pwr, (pwr >> 4) & 3, osr, odr, cfg);
}
inline void log_heap() {
ESP_LOGI(TAG, "HEAP internal_free=%u internal_total=%u min_free=%u",
(unsigned) heap_caps_get_free_size(MALLOC_CAP_INTERNAL),
(unsigned) heap_caps_get_total_size(MALLOC_CAP_INTERNAL),
(unsigned) heap_caps_get_minimum_free_size(MALLOC_CAP_INTERNAL));
}
// Probe the chip; on failure log the line levels, reset the IDF bus once and probe again.
inline void probe(uint8_t addr, int scl) {
uint8_t v = 0;
bool ok = rd(addr, 0x00, &v);
ESP_LOGI(TAG, "PROBE %s chip=0x%02X held=%d sel=0x%08X sda=%d scl=%d", ok ? "ok" : "err", v, g_held,
(unsigned) out_sel_reg(), gpio_get_level((gpio_num_t) g_sda), gpio_get_level((gpio_num_t) scl));
if (ok || !g_bus_reset_on_probe)
return;
i2c_master_bus_handle_t h;
esp_err_t e = i2c_master_get_bus_handle((i2c_port_num_t) g_port, &h);
if (e == ESP_OK)
e = i2c_master_bus_reset(h);
ok = rd(addr, 0x00, &v);
ESP_LOGI(TAG, "PROBE after bus reset (%d): %s chip=0x%02X", e, ok ? "ok" : "err", v);
}
} // namespace bmpt
@@ -1,208 +0,0 @@
"""Write a BMP390 bench config (ESP32-S3, no Wi-Fi) next to this script.
usage: gen.py NAME [--rate 50Hz] [--iir 128x|off] [--interval 1s] [--fault --kinds NMTWLSHD --probe-delay MS]
[--reset-at 40,80] [--open-line] [--bus-reset]
Fault kinds, one per event at 30 s and then every 20 s: N next start write NACKs in software, M every start write
NACKs for 5 s, T SDA held low for the next start write, W the same for every transfer for 5 s, L a 5 s continuous
hold, S a 300 ms hold, H and D harness-only held transfers.
"""
import argparse
import pathlib
HERE = pathlib.Path(__file__).parent
FAULT = """
- interval: 100ms
then:
- lambda: |-
bmpt::fault_tick();
static constexpr char KIND[] = "KINDS";
static uint8_t i = 0;
static uint32_t probe_at = 0;
uint32_t now = millis();
if (i < sizeof(KIND) - 1 && now >= 30000u + 20000u * i) { // events at 30 s, then every 20 s
char k = KIND[i++];
ESP_LOGI("bmpt", "FAULT %c at %u failed=%d warn=%d", k, (unsigned) now, id(bmp)->is_failed(),
id(bmp)->status_has_warning());
// S and L hold SDA at once; fault_tick() releases it at the deadline
if (k == 'S') {
bmpt::g_fb->hold_until = now + 300;
bmpt::sda_hold();
}
if (k == 'W') bmpt::g_fb->window_until = now + 5000;
if (k == 'L' || k == 'D') bmpt::g_fb->hold_until = now + 5000;
if (k == 'L')
bmpt::sda_hold();
if (k == 'N') bmpt::g_fb->soft_arm = true;
if (k == 'M') bmpt::g_fb->soft_until = now + 5000;
if (k == 'H') {
uint8_t reg = 0x00, v;
bmpt::g_fb->arm_start = false;
bmpt::sda_hold();
bool hok = bmpt::g_real->write_readv(ADDR, &reg, 1, &v, 1) == esphome::i2c::ERROR_OK;
bmpt::sda_release();
ESP_LOGI("bmpt", "HARNESS held transfer ok=%d", hok);
}
if (k == 'D') {
uint8_t reg = 0x00, v;
for (int n = 0; n < 2; n++)
bmpt::g_fb->write_readv(ADDR, &reg, 1, &v, 1);
}
if (k == 'T') bmpt::g_fb->arm_start = true;
probe_at = now + PROBEDELAY;
}
if (probe_at != 0 && now >= probe_at) {
probe_at = 0;
bmpt::probe(ADDR, SCLPIN);
}
"""
RESET = """
- interval: 1s
then:
- lambda: |-
static constexpr uint32_t AT[] = {RESETAT};
static uint8_t i = 0;
if (i < sizeof(AT) / sizeof(AT[0]) && millis() >= AT[i] * 1000) {
i++;
bool ok = bmpt::wr(ADDR, 0x7E, 0xB6);
delay(10);
ESP_LOGI("bmpt", "SOFTRESET ok=%d", ok);
bmpt::dump_regs(ADDR);
ESP_LOGI("bmpt", "DRDY ok=%d", bmpt::wr(ADDR, 0x19, 0x42));
}
"""
def main() -> None:
ap = argparse.ArgumentParser()
ap.add_argument("name")
ap.add_argument("--addr", default="0x77")
ap.add_argument("--rate")
ap.add_argument("--iir", default="128x")
ap.add_argument("--interval", default="1s")
ap.add_argument("--posr", default="8x")
ap.add_argument("--tosr", default="none")
ap.add_argument("--fault", action="store_true")
ap.add_argument("--kinds", default="WTSWTSWT")
ap.add_argument("--probe-delay", default="7000")
ap.add_argument(
"--open-line",
action="store_true",
help="fault routes SDA away (floats high) instead of low",
)
ap.add_argument(
"--bus-reset",
action="store_true",
help="reset the IDF bus after a failed probe",
)
ap.add_argument("--sda", default="1")
ap.add_argument("--scl", default="2")
ap.add_argument("--int-pin", default="42")
ap.add_argument(
"--no-drdy", action="store_true", help="do not enable data ready on INT"
)
ap.add_argument(
"--reset-at",
default="",
help="seconds, comma separated: soft reset the chip behind the driver",
)
a = ap.parse_args()
rate = f"\n data_rate: {a.rate}" if a.rate else ""
# data ready on INT, push-pull, active high; the driver's soft reset in setup clears it
drdy = (
""
if a.no_drdy
else f' ESP_LOGI("bmpt", "DRDY ok=%d", bmpt::wr({a.addr}, 0x19, 0x42));\n'
)
extra = ""
if a.fault:
extra += (
FAULT.replace("KINDS", a.kinds)
.replace("PROBEDELAY", a.probe_delay)
.replace("SCLPIN", a.scl)
)
if a.reset_at:
extra += RESET.replace("RESETAT", a.reset_at)
yaml = f"""esphome:
name: {a.name}
includes:
- bmp390probe.hpp
on_boot:
- priority: 800
then:
- lambda: |-
bmpt::g_real = id(bus);
bmpt::g_sda = {a.sda};
bmpt::g_port = id(bus)->get_port();
static bmpt::FaultBus fb(id(bus));
id(bmp)->set_i2c_bus(&fb);
bmpt::g_fb = &fb;
bmpt::g_bus_reset_on_probe = {"true" if a.bus_reset else "false"};
bmpt::g_hold_level = {1 if a.open_line else 0};
ESP_LOGI("bmpt", "PRESETUP");
bmpt::dump_regs({a.addr});
- priority: -100
then:
- lambda: |-
ESP_LOGI("bmpt", "POSTSETUP failed=%d", id(bmp)->is_failed());
bmpt::dump_regs({a.addr});
{drdy} bmpt::int_counter_begin({a.int_pin});
bmpt::dump_regs({a.addr});
bmpt::log_heap();
esp32:
board: esp32-s3-devkitc-1
framework:
type: esp-idf
logger:
level: DEBUG
hardware_uart: USB_SERIAL_JTAG
i2c:
id: bus
sda: GPIO{a.sda}
scl: GPIO{a.scl}
scan: true
sensor:
- platform: bmp3xx_i2c
id: bmp
address: {a.addr}
pressure:
name: pressure
oversampling: {a.posr}
on_value:
- lambda: 'ESP_LOGI("bmpt", "READ %u %.4f", (unsigned) millis(), x);'
temperature:
name: temperature
oversampling: {a.tosr}
iir_filter: "{a.iir}"{rate}
update_interval: {a.interval}
interval:
- interval: 10s
then:
- lambda: |-
static uint32_t last_edges = 0, last_ms = 0;
uint32_t e = bmpt::g_int_edges, now = millis();
ESP_LOGI("bmpt", "INT edges=%u in %u ms failed=%d warn=%d xfers=%u fails=%u starts=%u "
"start_fails=%u normal_writes=%u",
(unsigned) (e - last_edges), (unsigned) (now - last_ms), id(bmp)->is_failed(),
id(bmp)->status_has_warning(), bmpt::g_fb->xfers, bmpt::g_fb->fails, bmpt::g_fb->start_writes,
bmpt::g_fb->start_fails, bmpt::g_fb->normal_writes);
last_edges = e;
last_ms = now;
if (!bmpt::g_held) bmpt::dump_regs({a.addr});
bmpt::log_heap();
{extra.replace("ADDR", a.addr)}"""
out = HERE / f"{a.name}.yaml"
out.write_text(yaml, encoding="utf-8")
print(out)
if __name__ == "__main__":
main()
@@ -1,15 +0,0 @@
#!/bin/bash
# run.sh NAME SECONDS [TAG]: build NAME.yaml (written by gen.py) from TREE, flash the ESP32-S3, capture SECONDS from
# a fresh boot into out-NAME/TAG.txt and print stats.py's summary.
# env: TREE (esphome checkout), PORT (serial port), SC (path to serial_capture.py), PY (checkout python)
R=$(cd "$(dirname "$0")" && pwd); N=${1:?name}; S=${2:?seconds}; G=${3:-run1}
T=${TREE:?set TREE}; P=${PORT:?set PORT}; PY=${PY:-$T/venv/bin/python}; SC=${SC:?set SC}
[ -f "$R/$N.yaml" ] || { echo "no $N.yaml, run gen.py first"; exit 1; }
OUT=$R/out-$N; mkdir -p "$OUT"
pkill -f "[s]erial_capture.py $P"
(cd "$T" && $PY -m esphome run --no-logs "$R/$N.yaml" --device "$P" > "$OUT/upload.log" 2>&1) ||
{ echo "build or upload failed"; tail -5 "$OUT/upload.log"; exit 1; }
grep -a -E '^(RAM|Flash):' "$OUT/upload.log"
$PY "$SC" "$P" "$S" "$OUT/$G.txt" --reset > /dev/null 2>&1
[ -s "$OUT/$G.txt" ] || { echo "serial capture got nothing"; exit 1; }
$PY "$R/stats.py" "$OUT/$G.txt"
@@ -1,63 +0,0 @@
"""Pressure statistics from a bmp390 capture.
usage: stats.py CAPTURE [SKIP_S]
Prints the read count and cadence, the noise (standard deviation, of consecutive differences and detrended over a
21 sample window), identical consecutive readings, the INT data ready rate per 10 s window and counts of errors,
warnings and harness lines. SKIP_S drops the readings before that many seconds after boot (IIR settling).
"""
import pathlib
import re
import statistics
import sys
def main() -> None:
lines = pathlib.Path(sys.argv[1]).read_bytes().decode(errors="replace").splitlines()
skip = float(sys.argv[2]) if len(sys.argv) > 2 else 0
reads = [
(int(m[1]), float(m[2]))
for line in lines
if (m := re.search(r"READ (\d+) ([\d.]+)", line))
]
r = [p for t, p in reads if t / 1000 >= skip]
if len(r) > 2:
d = [b - a for a, b in zip(r, r[1:], strict=False)]
same = sum(1 for x in d if x == 0)
print(
f"reads={len(reads)} used={len(r)} mean={statistics.mean(r):.4f} sd={statistics.pstdev(r) * 100:.2f}Pa "
f"sd_diff={statistics.pstdev(d) * 100:.2f}Pa identical_consecutive={same} min={min(r):.4f} max={max(r):.4f}"
)
ts = [t for t, _ in reads]
if len(ts) > 2:
gaps = [b - a for a, b in zip(ts, ts[1:], strict=False)]
print(f"cadence mean={statistics.mean(gaps):.0f}ms max_gap={max(gaps)}ms")
ints = [
(int(m[1]), int(m[2]))
for line in lines
if (m := re.search(r"INT edges=(\d+) in (\d+) ms", line))
]
# the first window starts at boot, before data ready is on
if rates := [e * 1000 / ms for e, ms in ints[1:] if ms > 9000]:
print(f"INT rate Hz: {' '.join(f'{x:.2f}' for x in rates)}")
for pat in (
r"\[E\]",
r"\[W\]",
"XFERFAIL",
"SOFTFAIL",
"FAULT",
"PROBE",
"marked FAILED",
"REGS",
):
print(f"{pat}: {sum(1 for line in lines if re.search(pat, line))}")
if len(r) > 30:
k = 10
res = [
r[i] - statistics.mean(r[i - k : i + k + 1]) for i in range(k, len(r) - k)
]
print(f"detrended sd (21 sample window)={statistics.pstdev(res) * 100:.3f}Pa")
if __name__ == "__main__":
main()
@@ -1,170 +0,0 @@
# CC1101 sleep and wake checks T0 to T6 on an ESP32-S3 (SPI on GPIO12/11/13, CS GPIO10, GDO0 GPIO9), run 4 s after
# boot; the result is the "DONE fails=N" line. Substitution: name.
esphome:
name: ${name}
includes: [cctest.hpp]
on_boot:
- priority: -100
then:
- delay: 4s
- script.execute: run_tests
esp32:
variant: esp32s3
framework:
type: esp-idf
logger:
level: DEBUG
hardware_uart: USB_SERIAL_JTAG
spi:
clk_pin: GPIO12
mosi_pin: GPIO11
miso_pin: GPIO13
cc1101:
id: transceiver
cs_pin: GPIO10
gdo0_pin: GPIO9
frequency: 433.92MHz
modulation_type: GFSK
symbol_rate: 4800
packet_mode: true
packet_length: 8
crc_enable: true
sync_mode: "16/16"
output_power: 0
on_packet:
then:
- lambda: ESP_LOGI("cct", "RX packet %s rssi %.1f", format_hex(x).c_str(), rssi);
script:
- id: run_tests
then:
- lambda: |-
using namespace cct;
auto *c = id(transceiver);
Snap base, s;
int fails = 0;
snap(c, base);
ESP_LOGI("cct", "T0 baseline MARCSTATE=%02X FREQ=%02X%02X%02X TEST2=%02X TEST1=%02X TEST0=%02X so_high_awake=%d",
base.marc, base.reg[0x0D], base.reg[0x0E], base.reg[0x0F], base.reg[0x2C], base.reg[0x2D],
base.reg[0x2E], cs_low_so_high(c));
hexpa("T0 baseline", base.pa);
// the sleep checks below need a probe that sees an awake chip as awake
if (cs_low_so_high(c)) {
ESP_LOGW("cct", "T0 FAIL: SO reads high on an awake chip, the sleep checks cannot be trusted");
fails++;
}
// T1: mark PATABLE 1-7, sleep, wake by hand: lost entries prove the chip slept
const uint8_t marks[8] = {base.pa[0], 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77};
write_pa(c, marks);
snap(c, s); hexpa("T1 marked", s.pa);
c->power_down();
bool so = cs_low_so_high(c);
snap(c, s);
hexpa("T1 after raw wake", s.pa);
bool lost = s.pa[1] != 0x11;
ESP_LOGI("cct", "T1 sleep proof: so_high_on_cs=%d pa_lost=%d MARCSTATE=%02X TEST0=%02X -> %s", so, lost, s.marc,
s.reg[0x2E], (so && lost) ? "PASS" : "FAIL");
if (!(so && lost)) fails++;
// the component still thinks it sleeps
c->wake_up();
snap(c, s);
int d = diff(base, s, "T1 restore");
ESP_LOGI("cct", "T1 restore after wake_up: %d differences, MARCSTATE=%02X -> %s", d, s.marc, d == 0 ? "PASS" : "FAIL");
if (d) fails++;
c->begin_rx();
// T2: change settings while asleep, compare with the same change made awake
Snap awake;
c->set_frequency(433.42e6); c->set_output_power(5); c->set_symbol_rate(9600);
snap(c, awake);
c->set_frequency(433.92e6); c->set_output_power(0); c->set_symbol_rate(4800);
c->power_down();
c->set_frequency(433.42e6); c->set_output_power(5); c->set_symbol_rate(9600);
bool so2 = cs_low_so_high(c); // wakes the chip by hand; component still asleep
c->wake_up();
c->begin_rx();
snap(c, s);
awake.marc = s.marc;
d = diff(awake, s, "T2 asleep vs awake");
ESP_LOGI("cct", "T2 settings changed asleep: so_high=%d %d differences FREQ=%02X%02X%02X MARCSTATE=%02X -> %s", so2, d,
s.reg[0x0D], s.reg[0x0E], s.reg[0x0F], s.marc, (so2 && d == 0) ? "PASS" : "FAIL");
if (!(so2 && d == 0)) fails++;
c->set_frequency(433.92e6); c->set_output_power(0); c->set_symbol_rate(4800);
c->begin_rx();
snap(c, base);
// T3: send while asleep wakes the radio by itself
c->power_down();
uint8_t pkt[8] = {1, 2, 3, 4, 5, 6, 7, 8};
int err = (int) c->transmit_packet(pkt, sizeof(pkt));
snap(c, s);
d = diff(base, s, "T3 after send");
ESP_LOGI("cct", "T3 send while asleep: err=%d MARCSTATE=%02X TXBYTES=%02X %d differences -> %s", err, s.marc,
status(c, 0x3A), d, (err == 0 && d == 0) ? "PASS" : "FAIL");
if (!(err == 0 && d == 0)) fails++;
// T4: begin_rx while asleep
c->power_down();
c->begin_rx();
snap(c, s);
d = diff(base, s, "T4 begin_rx");
ESP_LOGI("cct", "T4 begin_rx while asleep: MARCSTATE=%02X %d differences -> %s", s.marc, d,
(s.marc == 0x0D && d == 0) ? "PASS" : "FAIL");
if (!(s.marc == 0x0D && d == 0)) fails++;
// T5: 200 cycles, each one proven asleep and restored
int bad = 0, notslept = 0;
uint32_t t0 = micros();
for (int i = 0; i < 200; i++) {
write_pa(c, marks);
c->power_down();
if (!cs_low_so_high(c)) notslept++;
c->wake_up();
c->begin_rx();
snap(c, s);
if (diff(base, s, "T5") != 0) bad++;
if ((i & 31) == 0) App.feed_wdt();
}
ESP_LOGI("cct", "T5 200 cycles: not_slept=%d restore_fail=%d in %u ms heap=%u -> %s", notslept, bad,
(unsigned) ((micros() - t0) / 1000), (unsigned) esp_get_free_heap_size(),
(notslept == 0 && bad == 0) ? "PASS" : "FAIL");
if (notslept || bad) fails++;
// T6: each setter on its own while asleep: does it wake the chip, and in which state does it leave it
struct Case { const char *name; int kind; };
const Case cases[] = {{"set_frequency", 0}, {"set_channel", 1}, {"set_modulation_type", 2}, {"set_symbol_rate", 3},
{"set_output_power", 4}, {"set_filter_bandwidth", 5}, {"set_fsk_deviation", 6}};
for (auto &k : cases) {
c->begin_rx();
c->power_down();
switch (k.kind) {
case 0: c->set_frequency(433.92e6); break;
case 1: c->set_channel(0); break;
case 2: c->set_modulation_type(cc1101::Modulation::MODULATION_GFSK); break;
case 3: c->set_symbol_rate(4800); break;
case 4: c->set_output_power(0); break;
case 5: c->set_filter_bandwidth(203e3); break;
case 6: c->set_fsk_deviation(1.5e3); break;
}
bool asleep = cs_low_so_high(c);
uint8_t marc = status(c, 0x35) & 0x1F;
ESP_LOGI("cct", "T6 %-20s while asleep: chip %s, MARCSTATE=%02X", k.name, asleep ? "still asleep" : "AWAKE", marc);
if (!asleep) fails++;
c->wake_up();
}
c->begin_rx();
uint8_t agc_rx = rd(c, 0x2B);
c->set_idle();
uint8_t agc_idle = rd(c, 0x2B);
ESP_LOGI("cct", "AGCTEST in RX=%02X in IDLE=%02X", agc_rx, agc_idle);
c->begin_rx();
ESP_LOGI("cct", "DONE fails=%d", fails);
button:
- platform: template
id: yaml_actions
name: Yaml actions
on_press:
- cc1101.power_down: transceiver
- cc1101.wake_up: transceiver
- cc1101.power_down: transceiver
- cc1101.send_packet: [0x12, 0x34, 0x56, 0x78, 0x90, 0xab, 0xcd, 0xef]
- lambda: ESP_LOGI("cct", "YAML actions done MARCSTATE=%02X", cct::status(id(transceiver), 0x35) & 0x1F);
@@ -1,70 +0,0 @@
// Register and PATABLE snapshots plus the CHIP_RDYn sleep check for the cc1101 sleep tests in cc1101-sleep.yaml.
#pragma once
#include "esphome/components/cc1101/cc1101.h"
#include "driver/gpio.h"
#include <cstring>
namespace cct {
using esphome::cc1101::CC1101Component;
static constexpr gpio_num_t MISO = GPIO_NUM_13; // the SPI MISO pin in cc1101-sleep.yaml
static constexpr uint8_t NREG = 0x2F; // IOCFG2..TEST0
struct Snap {
uint8_t reg[NREG];
uint8_t pa[8];
uint8_t marc;
};
// Chip ready flag: with CSn low, SO stays high until the crystal runs (SLEEP), low at once otherwise
inline bool cs_low_so_high(CC1101Component *c) {
c->enable();
bool high = gpio_get_level(MISO);
delayMicroseconds(300);
c->disable();
return high;
}
inline uint8_t rd(CC1101Component *c, uint8_t a) {
c->enable();
c->write_byte(a | 0x80);
uint8_t v = c->read_byte();
c->disable();
return v;
}
inline uint8_t status(CC1101Component *c, uint8_t a) { return rd(c, a | 0x40); }
inline void snap(CC1101Component *c, Snap &s) {
for (uint8_t i = 0; i < NREG; i++)
s.reg[i] = rd(c, i);
c->enable();
c->write_byte(0x3E | 0xC0);
for (auto &b : s.pa)
b = c->read_byte();
c->disable();
s.marc = status(c, 0x35) & 0x1F;
}
inline void write_pa(CC1101Component *c, const uint8_t *pa) {
c->enable();
c->write_byte(0x3E | 0x40);
for (int i = 0; i < 8; i++)
c->write_byte(pa[i]);
c->disable();
}
inline int diff(const Snap &a, const Snap &b, const char *what) {
int n = 0;
for (uint8_t i = 0; i < NREG; i++)
// FSCAL3..FSCAL1 hold calibration results, FSTEST and AGCTEST are chip owned test registers
if (i != 0x23 && i != 0x24 && i != 0x25 && i != 0x29 && i != 0x2B && a.reg[i] != b.reg[i]) {
ESP_LOGW("cct", "%s reg 0x%02X: %02X vs %02X", what, i, a.reg[i], b.reg[i]);
n++;
}
for (int i = 0; i < 8; i++)
if (a.pa[i] != b.pa[i]) {
ESP_LOGW("cct", "%s pa[%d]: %02X vs %02X", what, i, a.pa[i], b.pa[i]);
n++;
}
return n;
}
inline void hexpa(const char *tag, const uint8_t *pa) {
ESP_LOGI("cct", "%s PA %02X %02X %02X %02X %02X %02X %02X %02X", tag, pa[0], pa[1], pa[2], pa[3], pa[4], pa[5], pa[6],
pa[7]);
}
} // namespace cct
@@ -1,102 +0,0 @@
"""quiet_then_subscribe.py HOST PORT OUT QUIET_BOOTS: on the first boot drain any crash record (the first log
subscribe replays and clears it), then stay connected without a log subscription for QUIET_BOOTS reboots, then
subscribe on the next boot and record the log (including a crash record replayed on subscribe) until a line with
REBOOT. The drain connection is dropped and replaced by one without a subscription before boot 1 reboots, so a crash
that needs a log subscriber is not recorded as a quiet one; a reboot during the drain, or a reconnect slow enough to
hide one, fails the run instead of shifting the boot count. Exits 1 when that line never comes or a quiet boot does
not reboot within QUIET_REBOOT_WAIT seconds (default 900). Every lost connection counts as a reboot, so run it on a
stable link.
env: API_KEY (unset for a plaintext API), QUIET_REBOOT_WAIT."""
import asyncio
import contextlib
import os
import pathlib
import sys
import time
from aioesphomeapi import LogLevel
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from apiconn import connect, log_text, stop_event # noqa: E402
# how long a quiet boot may run before its reboot; QUIET_REBOOT_WAIT in the env overrides it
REBOOT_WAIT = float(os.environ.get("QUIET_REBOOT_WAIT", "900"))
# a reconnect slower than this may have waited out a reboot (connect() retries once a second)
RECONNECT_MAX_S = 5.0
async def main(host: str, port: int, outf: str, quiet: int) -> None:
with pathlib.Path(outf).open("a", encoding="utf-8") as f:
def w(t: str) -> None:
f.write(f"{time.strftime('%H:%M:%S')} {t}\n")
f.flush()
boots = 0
while True:
lost, on_stop = stop_event()
cli = await connect(host, tries=300, timeout=5, port=port, on_stop=on_stop)
boots += 1
info = await cli.device_info()
w(
f"=== CONNECTED boot {boots} name={info.name} quiet={1 < boots <= quiet + 1}"
)
if boots > quiet + 1:
done = asyncio.Event()
def on_log(m, done: asyncio.Event = done) -> None:
t = log_text(m)
w(t)
if "REBOOT" in t:
done.set()
cli.subscribe_logs(
on_log, log_level=LogLevel.LOG_LEVEL_DEBUG, dump_config=False
)
try:
await asyncio.wait_for(done.wait(), 120)
except TimeoutError:
w("=== no REBOOT within 120 s")
with contextlib.suppress(Exception):
await cli.disconnect()
if not done.is_set():
sys.exit("FAIL: no REBOOT line within 120 s of subscribing")
return
if boots == 1:
# drain any crash record left by the reboot that brought this build up
cli.subscribe_logs(
lambda m: w("drain " + log_text(m)),
log_level=LogLevel.LOG_LEVEL_DEBUG,
dump_config=False,
)
await asyncio.sleep(3)
if lost.is_set():
w("=== boot 1 rebooted during the drain")
sys.exit("FAIL: boot 1 rebooted with the drain subscriber attached")
with contextlib.suppress(Exception):
await cli.disconnect()
lost, on_stop = stop_event()
t = time.monotonic()
cli = await connect(
host, tries=300, timeout=5, port=port, on_stop=on_stop
)
if (dt := time.monotonic() - t) > RECONNECT_MAX_S:
w(f"=== boot 1 reconnect took {dt:.1f} s")
sys.exit(
f"FAIL: the reconnect after the drain took {dt:.1f} s, boot 1 may have rebooted"
)
w("=== boot 1 reconnected without a log subscription")
# quiet: wait for the device to drop the connection (its reboot)
try:
await asyncio.wait_for(lost.wait(), REBOOT_WAIT)
except TimeoutError:
w(f"=== boot {boots}: no reboot within {REBOOT_WAIT:.0f} s")
sys.exit(
f"FAIL: boot {boots} did not reboot within {REBOOT_WAIT:.0f} s"
)
w("=== connection lost")
await asyncio.sleep(2)
asyncio.run(main(sys.argv[1], int(sys.argv[2]), sys.argv[3], int(sys.argv[4])))
@@ -1,15 +0,0 @@
esphome:
includes:
- crctest.hpp
on_boot:
- priority: -100
then:
- delay: 3s
- lambda: |-
// unqualified call, as a lambda user would write it; counted in the CRCRESULT totals
crctest::run(crc32(0, reinterpret_cast<const uint8_t *>("123456789"), 9));
interval:
- interval: 20s
then:
- lambda: crctest::run();
@@ -1,73 +0,0 @@
#pragma once
// crc32 vectors for esphome#22398, expected values from Python zlib.crc32
#include "esphome/core/helpers.h"
#include "esphome/core/hal.h"
#include "esphome/core/log.h"
#include <cstdlib>
#include <cstring>
namespace crctest {
inline int pass_n = 0, fail_n = 0;
inline void check(const char *name, uint32_t got, uint32_t want) {
if (got == want)
pass_n++;
else
fail_n++;
ESP_LOGI("crctest", "%s %s got=%08X want=%08X", got == want ? "PASS" : "FAIL", name, (unsigned) got, (unsigned) want);
}
// unqualified: a crc32() of "123456789" computed in the caller's lambda, or -1 to skip that check
inline void run(int64_t unqualified = -1) {
pass_n = fail_n = 0;
const uint8_t *s = reinterpret_cast<const uint8_t *>("123456789");
check("check-123456789", esphome::crc32(0, s, 9), 0xCBF43926u);
check("empty-seed0", esphome::crc32(0, s, 0), 0u);
check("empty-seed12345678", esphome::crc32(0x12345678u, s, 0), 0x12345678u);
check("chained-4+5", esphome::crc32(esphome::crc32(0, s, 4), s + 4, 5), 0xCBF43926u);
uint8_t all[256];
for (int i = 0; i < 256; i++)
all[i] = i;
check("bytes-0-255", esphome::crc32(0, all, 256), 0x29058C73u);
uint8_t ff[32];
memset(ff, 0xFF, sizeof(ff));
check("32x0xFF", esphome::crc32(0, ff, 32), 0xFF6CAB0Bu);
uint8_t *pat = static_cast<uint8_t *>(malloc(4100 + 8192));
if (pat == nullptr) {
ESP_LOGE("crctest", "malloc failed");
fail_n++;
ESP_LOGI("crctest", "CRCRESULT pass=%d fail=%d", pass_n, fail_n);
return;
}
for (int i = 0; i < 4100; i++)
pat[i] = (i * 7 + 3) & 0xFF;
static const uint32_t EXP_OFFSET[4] = {0xDF4C2BA9u, 0x5D4B54F6u, 0x3B885695u, 0xAE989C6Eu};
for (int o = 0; o < 4; o++) {
char n[24];
snprintf(n, sizeof(n), "unaligned-off%d", o);
check(n, esphome::crc32(0, pat + o, 4093), EXP_OFFSET[o]);
}
uint32_t c = 0;
for (int i = 0; i < 300; i++)
c = esphome::crc32(c, pat + i, 1);
check("chained-1-byte-x300", c, 0xDE0E57CEu);
c = 0;
for (int i = 0; i < 4093; i += 97)
c = esphome::crc32(c, pat + 1 + i, std::min(97, 4093 - i));
check("chained-97-byte-unaligned", c, 0x5D4B54F6u);
uint8_t *big = pat + 4100;
for (int i = 0; i < 8192; i++)
big[i] = (i * 13 + 5) & 0xFF;
uint32_t best = 0xFFFFFFFF, r = 0;
for (int k = 0; k < 5; k++) {
const uint32_t t0 = (esphome::micros) ();
r = esphome::crc32(0, big, 8192);
const uint32_t dt = (esphome::micros) () - t0;
if (dt < best)
best = dt;
}
check("8KiB", r, 0xF6950930u);
free(pat);
ESP_LOGI("crctest", "CRCSPEED 8192 bytes in %u us = %.1f us/KiB", (unsigned) best, best / 8.0f);
if (unqualified >= 0)
check("unqualified", (uint32_t) unqualified, 0xCBF43926u);
ESP_LOGI("crctest", "CRCRESULT pass=%d fail=%d", pass_n, fail_n);
}
} // namespace crctest
@@ -1,168 +0,0 @@
"""trcheck.py D1HOST D1VAR E32HOST E32VAR SECONDS OUT.json: watch the entity_transport remote entities on both
boards over the API and compare each received value with the sender's own published values."""
import asyncio
import json
import math
import pathlib
import struct
import sys
import time
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from apiconn import connect # noqa: E402
D1, D1V, E32, E32V, SECS, OUT = (
sys.argv[1],
sys.argv[2],
sys.argv[3],
sys.argv[4],
float(sys.argv[5]),
sys.argv[6],
)
def f32(v):
return struct.unpack("<f", struct.pack("<f", v))[0]
def expect(t): # 12345.678f + t * 0.001f - (t % 7) * 1000.5f in single precision
return f32(f32(f32(12345.678) + f32(f32(t) * f32(0.001))) - f32((t % 7) * 1000.5))
async def main():
boards = {"d1": (D1, D1V), "e32": (E32, E32V)}
hist = {
b: {"count": [], "float": set(), "bin": [], "count_to_float": {}}
for b in boards
}
obs = {} # (receiver, kind, field) -> list of (t, value, missing)
clients = {}
for b, (host, _) in boards.items():
c = await connect(host)
clients[b] = c
ents, _ = await c.list_entities_services()
keys = {e.key: e.object_id for e in ents}
peer_var = boards["e32" if b == "d1" else "d1"][1]
last = {}
def cb(s, b=b, keys=keys, peer_var=peer_var, last=last):
oid = keys.get(s.key)
t = time.monotonic()
if oid is None:
return
miss = bool(s.missing_state) or (
isinstance(s.state, float) and math.isnan(s.state)
)
h = hist[b]
if oid == "local_count" and not miss:
h["count"].append((t, int(s.state)))
last["count"] = int(s.state)
elif oid == "local_float" and not miss:
h["float"].add(f32(s.state))
if "count" in last:
h["count_to_float"][last["count"]] = f32(s.state)
elif oid == "local_bin":
h["bin"].append((t, bool(s.state)))
elif oid.startswith("r_"):
_, kind, var, field = oid.split("_")
if var == peer_var:
obs.setdefault(f"{b}<-{kind}:{field}", []).append(
(t, s.state, miss)
)
elif not miss:
obs.setdefault(f"{b}<-{kind}:{field}:otherpeer", []).append(
(t, s.state, False)
)
c.subscribe_states(cb)
await asyncio.sleep(SECS)
for c in clients.values():
await c.disconnect()
res = {"d1": D1V, "e32": E32V, "seconds": SECS, "links": {}}
for key, lst in sorted(obs.items()):
recv, rest = key.split("<-")
kind, field = rest.split(":")[:2]
sender = "e32" if recv == "d1" else "d1"
h = hist[sender]
if key.endswith("otherpeer"):
res["links"][key] = {"values_after_initial": len(lst) - 1}
continue
vals = [(t, v) for t, v, m in lst if not m][
1:
] # the first is the state from before the run
missing_after_first = sum(
1
for i, (t, v, m) in enumerate(lst)
if m and any(not mm for _, _, mm in lst[:i])
)
r = {"updates": len(vals), "missing_after_first": missing_after_first}
if field == "count":
sent = {c for _, c in h["count"]}
first = min(sent) if sent else 0
r["before_run"] = sum(1 for _, v in vals if int(v) < first)
last_sent = max(sent) if sent else 0
r["not_sent"] = sum(
1 for _, v in vals if int(v) > last_sent + 60 or int(v) != v
)
r["api_gap_values"] = sum(
1
for _, v in vals
if first <= int(v) <= last_sent and int(v) not in sent
)
lags = []
for t, v in vals:
prior = [c for tt, c in h["count"] if tt <= t]
if prior:
lags.append(prior[-1] - int(v))
r["max_lag_s"] = max(lags) if lags else None
elif field == "flt" and not h["count"]:
# no sender count over the API: the floats cannot be checked, which fails the link
r["no_sender_counts"] = True
elif field == "flt":
ok_t = [
t
for t, v, _m in obs[key.replace("flt", "count")]
if not (isinstance(v, float) and math.isnan(v))
and int(v) >= min(c for _, c in h["count"])
]
t0 = min(ok_t) if ok_t else 0
lo, hi = min(c for _, c in h["count"]), max(c for _, c in h["count"]) + 60
allowed = {expect(c) for c in range(max(1, lo - 30), hi + 1)}
r["not_sent"] = sum(1 for t, v in vals if t >= t0 and f32(v) not in allowed)
r["sender_float_mismatch"] = sum(
1 for c, fv in h["count_to_float"].items() if fv != expect(c)
)
else:
sent = [v for _, v in h["bin"]]
r["transitions"] = sum(
1 for i in range(1, len(vals)) if vals[i][1] != vals[i - 1][1]
)
r["sender_transitions"] = sum(
1 for i in range(1, len(sent)) if sent[i] != sent[i - 1]
)
res["links"][key] = r
ok = True
for k, r in res["links"].items():
if k.endswith("otherpeer"):
ok = ok and r["values_after_initial"] == 0
continue
if (
r["updates"] < 3
or r.get("not_sent", 0)
or r.get("sender_float_mismatch", 0)
or r.get("no_sender_counts")
):
ok = False
if "transitions" in r and r["transitions"] < 2:
ok = False
res["expected_links_present"] = (
len([k for k in res["links"] if not k.endswith("otherpeer")]) == 12
)
res["pass"] = ok and res["expected_links_present"]
print(json.dumps(res, indent=1))
pathlib.Path(OUT).write_text(json.dumps(res, indent=1), encoding="utf-8")
return 0 if res["pass"] else 1
sys.exit(asyncio.run(main()))
@@ -1,20 +0,0 @@
# Shared ESP8266 base for the lock load DUTs: reset reason at boot, no Wi-Fi sleep, web_server for the HTTP load.
esphome:
name: ${name}
on_boot:
- priority: 800
then:
- lambda: ESP_LOGI("lock", "BOOT reason=%s info=%s", ESP.getResetReason().c_str(), ESP.getResetInfo().c_str());
esp8266:
board: ${board}
logger:
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
power_save_mode: none
api:
reboot_timeout: 0s
ota:
- platform: esphome
web_server:
port: 80
@@ -1,39 +0,0 @@
# remote_transmitter and remote_receiver on the same pin (GPIO12), no jumper: one NEC frame every 250 ms, each one
# checked by the receiver's edge interrupt. Substitutions: name, board.
substitutions:
board: d1_mini
packages:
base: !include base.yaml
globals:
- {id: sent, type: uint32_t, initial_value: "0"}
- {id: rx_ok, type: uint32_t, initial_value: "0"}
- {id: rx_bad, type: uint32_t, initial_value: "0"}
remote_transmitter:
id: irtx
pin:
number: GPIO12
allow_other_uses: true
carrier_duty_percent: 100%
remote_receiver:
pin:
number: GPIO12
allow_other_uses: true
on_nec:
then:
- lambda: |-
if (x.address == 0x1234 && x.command == (uint16_t) (0x7800 | (id(sent) & 0xFF))) id(rx_ok)++; else id(rx_bad)++;
interval:
- interval: 250ms
then:
# The receiver's setup leaves the shared pin as an input; drive it again before each frame
- lambda: |-
pinMode(12, OUTPUT);
id(sent)++;
- remote_transmitter.transmit_nec:
address: 0x1234
command: !lambda return (uint16_t) (0x7800 | (id(sent) & 0xFF));
- interval: 10s
then:
- lambda: |-
ESP_LOGI("lock", "ALIVE up=%u sent=%u rx_ok=%u rx_bad=%u heap=%u", (unsigned) (millis() / 1000),
(unsigned) id(sent), (unsigned) id(rx_ok), (unsigned) id(rx_bad), (unsigned) ESP.getFreeHeap());
@@ -1,43 +0,0 @@
# DS18B20 on GPIO4 with a bus search every ${search_interval}. Set busy_control to "true" to replace the search with
# the same busy time and no 1-Wire traffic (the control run). Substitutions: name, board, search_interval, busy_control.
substitutions:
board: d1_mini
search_interval: 100ms
busy_control: "false"
packages:
base: !include base.yaml
one_wire:
- platform: gpio
pin: GPIO4
id: owbus
globals:
- {id: searches, type: uint32_t, initial_value: "0"}
- {id: search_bad, type: uint32_t, initial_value: "0"}
- {id: reads_ok, type: uint32_t, initial_value: "0"}
- {id: reads_nan, type: uint32_t, initial_value: "0"}
sensor:
- platform: dallas_temp
one_wire_id: owbus
id: t1
name: Temp
update_interval: 2s
on_raw_value:
- lambda: if (std::isnan(x)) id(reads_nan)++; else id(reads_ok)++;
interval:
- interval: ${search_interval}
then:
- lambda: |-
if (${busy_control}) {
delayMicroseconds(13000); // about one search with one sensor answering
return;
}
id(owbus)->search();
id(searches)++;
if (id(owbus)->get_devices().size() != 1)
id(search_bad)++;
- interval: 10s
then:
- lambda: |-
ESP_LOGI("lock", "ALIVE up=%u ok=%u nan=%u searches=%u search_bad=%u heap=%u t=%.2f",
(unsigned) (millis() / 1000), (unsigned) id(reads_ok), (unsigned) id(reads_nan),
(unsigned) id(searches), (unsigned) id(search_bad), (unsigned) ESP.getFreeHeap(), id(t1).state);
@@ -1,47 +0,0 @@
#!/bin/bash
# run.sh NAME SECONDS PPS: build NAME.yaml (next to this script, or a path) from TREE, flash it over serial, capture
# the log for SECONDS plus 160 (boot, up to 120 s for the IP, the load) and run wifi_load.py at PPS packets per second
# (0: no load). Prints the boot count, every reset reason and the last ALIVE line. Exit 0: a clean run (exactly one
# BOOT line); 1: the run finished and saw resets; 2: setup failed or the run is invalid (no load, capture cut short,
# or the sensor or IR loopback never worked).
# env: TREE (checkout under test), PORT, SC (path to serial_capture.py), PY (checkout python)
D=$(cd "$(dirname "$0")" && pwd); N=${1:?name}; T=${2:?seconds}; P=${3:?pps}
W=${TREE:?set TREE}; PORT=${PORT:?set PORT}; SC=${SC:?set SC}; PY=${PY:-$W/venv/bin/python}
Y=$N; [ -f "$Y" ] || Y=$D/$N.yaml; Y=$(cd "$(dirname "$Y")" && pwd)/$(basename "$Y"); N=$(basename "$Y" .yaml)
O=$D/out-$N; mkdir -p "$O"
pkill -f "[s]erial_capture.py $PORT "
rm -f "$O/serial.txt" "$O/load.txt"
(cd "$W" && PYTHONPATH=$W $PY -m esphome compile "$Y") > "$O/compile.log" 2>&1 || { echo "$N compile failed"; exit 2; }
(cd "$W" && PYTHONPATH=$W $PY -m esphome upload "$Y" --device "$PORT") > "$O/upload.log" 2>&1 || { echo "$N upload failed"; exit 2; }
$PY "$SC" "$PORT" $((T + 160)) "$O/serial.txt" --reset > "$O/capture.log" 2>&1 &
CAP=$!
for _ in $(seq 60); do
IP=$(LC_ALL=C grep -a -o -E "IP Address: [0-9.]+" "$O/serial.txt" | tail -1 | cut -d' ' -f3)
[ -n "$IP" ] && break; sleep 2
done
if [ "$P" -gt 0 ]; then
[ -n "$IP" ] || { kill "$CAP"; echo "$N no IP (check the secrets file)"; exit 2; }
python3 "$D/wifi_load.py" "$IP" "$T" "$P" > "$O/load.txt" 2>&1 || { echo "$N INVALID: load failed (see $O/load.txt)"; BAD=1; }
else
sleep "$T"
fi
wait "$CAP" || { echo "$N INVALID: capture failed (see $O/capture.log)"; BAD=1; }
BOOTS=$(LC_ALL=C grep -a -c 'BOOT reason' "$O/serial.txt")
echo "== $N pps=$P boots=$BOOTS"
LC_ALL=C grep -a "BOOT reason" "$O/serial.txt" | cut -c1-150
LC_ALL=C grep -a "ALIVE" "$O/serial.txt" | tail -1 | cut -c1-200
tail -1 "$O/load.txt" 2>/dev/null
ALIVE=$(LC_ALL=C grep -a "ALIVE" "$O/serial.txt" | tail -1)
# the workload must have run: a sensor answering searches, or decoded IR frames
case "$ALIVE" in
*" ok=0 "*|*" rx_ok=0 "*|"") echo "$N INVALID: no ALIVE line or the workload did not run"; BAD=1 ;;
esac
# one-wire: the searches must find the sensor (busy_control runs no searches)
S=$(echo "$ALIVE" | sed -n 's/.* searches=\([0-9]*\) search_bad=\([0-9]*\).*/\1 \2/p')
if [ -n "$S" ]; then
set -- $S
[ "$1" -gt 0 ] && [ "$2" -ge "$1" ] && { echo "$N INVALID: no search found the sensor"; BAD=1; }
fi
# resets are a failure even when they also broke the load or the workload
[ "$BOOTS" -gt 1 ] && exit 1
[ -z "$BAD" ] && [ "$BOOTS" = 1 ] || exit 2
@@ -1,60 +0,0 @@
"""Wi-Fi load on an ESP8266 under test: a fixed rate UDP flood to port 9 plus back to back web_server GETs.
usage: wifi_load.py HOST SECONDS PPS
Prints the counts every minute and a final "done" line.
"""
import socket
import sys
import threading
import time
import urllib.request
host, secs, pps = sys.argv[1], float(sys.argv[2]), int(sys.argv[3])
end = time.monotonic() + secs
stats = {"udp": 0, "udp_err": 0, "http_ok": 0, "http_err": 0, "http_gap_s": 0}
last_ok = [time.monotonic()]
def http_load() -> None:
while time.monotonic() < end:
try:
with urllib.request.urlopen(f"http://{host}/", timeout=5) as r:
r.read()
stats["http_ok"] += 1
now = time.monotonic()
stats["http_gap_s"] = max(stats["http_gap_s"], round(now - last_ok[0]))
last_ok[0] = now
except Exception: # noqa: BLE001 any failure counts; the thread must not die
stats["http_err"] += 1
time.sleep(1)
threading.Thread(target=http_load, daemon=True).start()
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
payload = b"x" * 512
nxt = last = time.monotonic()
while (now := time.monotonic()) < end:
try:
s.sendto(payload, (host, 9))
stats["udp"] += 1
except OSError: # mostly ENOBUFS when the host outruns the network
stats["udp_err"] += 1
time.sleep(0.01)
nxt += 1 / pps
if (d := nxt - time.monotonic()) > 0:
time.sleep(d)
if now - last > 60:
last = now
print(time.strftime("%H:%M:%S"), stats, flush=True)
stats["http_gap_s"] = max(stats["http_gap_s"], round(time.monotonic() - last_ok[0]))
print("done", stats, flush=True)
if (
stats["http_ok"] == 0
or stats["http_gap_s"] > 60 # the device stopped answering for over a minute
or stats["http_err"] > stats["http_ok"]
or stats["udp"] < secs * pps / 2
):
sys.exit("the load did not reach the device; the run is invalid")
@@ -9,7 +9,6 @@ before the LinkChecks instead.
import sys
import bk7231tools.serial.main as bk_main
from lt_platform import with_platform
from ltchiptool.__main__ import cli
from ltchiptool.soc.bk72xx.flash import BK72XXFlash
from serial import Serial
@@ -48,5 +47,5 @@ def flash_connect(self) -> None:
BK72XXFlash.flash_hw_reset = flash_hw_reset
BK72XXFlash.flash_connect = flash_connect
sys.argv = ["ltchiptool", *with_platform(sys.argv[1:])]
sys.argv = ["ltchiptool", *sys.argv[1:]]
cli()
@@ -11,7 +11,6 @@ Usage: python flash_rtl87xx_kit.py flash write -d <port> firmware.uf2
import sys
import time
from lt_platform import with_platform
import serial
_first = [True]
@@ -62,9 +61,7 @@ def boot_app(port: str) -> None:
def main() -> int:
rc = 1
try:
result = cli_entrypoint.main(
args=with_platform(sys.argv[1:]), standalone_mode=False
)
result = cli_entrypoint.main(args=sys.argv[1:], standalone_mode=False)
rc = int(result) if isinstance(result, int) else 0
finally:
port = port_argument(sys.argv)
@@ -1,62 +0,0 @@
"""cmp.py REF.json OTHER.json : list each UUID (service / characteristic / descriptor, in order) where OTHER differs
from REF, whether reads match (static characteristics), whether every notifying characteristic notified and
whether OTHER lost its link before the final disconnect. Exits 1 on any difference."""
import json
import pathlib
import sys
def flat(p):
d = json.loads(pathlib.Path(p).read_text(encoding="utf-8"))
rows = []
for s in d["services"]:
rows.append(("svc", s["uuid"].lower()))
for c in s["characteristics"]:
rows.append(("chr", c["uuid"].lower()))
rows += [("dsc", x.lower()) for x in c["descriptors"]]
return d, rows
rd, ref = flat(sys.argv[1])
od, oth = flat(sys.argv[2])
diff = [(a, b) for a, b in zip(ref, oth, strict=False) if a != b]
print(
f"entries ref={len(ref)} other={len(oth)} differing={len(diff) + abs(len(ref) - len(oth))}"
)
for a, b in diff:
print(f" {a[0]} ref {a[1]} other {b[1]}")
def chars(d: dict) -> list[dict]:
return [c for s in d["services"] for c in s["characteristics"]]
# compare by position: a UUID form bug renames the keys, so values and notify counts are matched by table order
problems = []
rc, oc = chars(rd), chars(od)
for i, (a, b) in enumerate(zip(rc, oc, strict=False)):
if a["props"] != b["props"]:
problems.append(
f"props {i} {b['uuid']} 0x{b['props']:02X} != ref 0x{a['props']:02X}"
)
rv, ov = rd["reads"].get(a["uuid"]), od["reads"].get(b["uuid"])
if str(ov).startswith("FAILED"):
problems.append(f"read {i} {b['uuid']} {ov}")
elif not a["props"] & 0x10 and rv is not None and ov != rv:
problems.append(f"read {i} {b['uuid']} {ov!r} != ref {rv!r}")
if a["props"] & 0x10:
n = od["notify"].get(b["uuid"])
if not isinstance(n, int) or n <= 0:
problems.append(f"notify {i} {b['uuid']} {n}")
# a link that dropped after the first notification still has counts, so the drop itself fails the run
if od.get("lost"):
problems.append(f"link lost before the disconnect: {od['lost']}")
print(f"notify {od['notify']}")
for x in problems:
print(" ", x)
if diff or len(ref) != len(oth) or problems:
sys.exit(
f"FAIL: {len(diff)} differing entries, {len(problems)} read or notify problems"
)
print("PASS")
@@ -1,100 +0,0 @@
"""List every service, characteristic and descriptor UUID a bluetooth_proxy reports for a peer, read each readable
characteristic, enable notify (CCCD write) on each notifying one and count 10 s of notifications, then disconnect.
A link lost before that disconnect is recorded as "lost" in the JSON, which cmp.py fails on.
usage: gattlist.py PROXY PEER_MAC OUT_JSON env: API_KEY (unset for a plaintext API). Retries 0x3E connect failures."""
import asyncio
import json
import pathlib
import sys
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from apiconn import ble_connect, connect, enable_notify # noqa: E402
async def once(host: str, addr: int) -> dict | None:
cli = await connect(host)
lost: list[int] = []
closing = False
def on_lost(e: int) -> None:
if not closing:
lost.append(e)
print(f"LINK LOST err={e}", flush=True)
c, _mtu, err, cancel = await ble_connect(cli, addr, on_lost)
if not c:
print(f"connect err={err}", flush=True)
cancel()
await cli.disconnect()
return None
svcs = await cli.bluetooth_gatt_get_services(addr)
out = {"services": [], "reads": {}, "notify": {}}
notif: dict[str, int] = {}
for s in svcs.services:
so = {"uuid": s.uuid, "characteristics": []}
for ch in s.characteristics:
so["characteristics"].append(
{
"uuid": ch.uuid,
"props": ch.properties,
"descriptors": [d.uuid for d in ch.descriptors],
}
)
if ch.properties & 0x02:
try:
out["reads"][ch.uuid] = bytes(
await cli.bluetooth_gatt_read(addr, ch.handle)
).hex()
except Exception as e: # noqa: BLE001
out["reads"][ch.uuid] = f"FAILED {type(e).__name__}"
if ch.properties & 0x10:
notif[ch.uuid] = 0
try:
if not await enable_notify(
cli,
addr,
ch,
lambda h, d, u=ch.uuid: (
notif.__setitem__(u, notif[u] + 1)
if isinstance(notif[u], int)
else None
),
):
notif[ch.uuid] = "FAILED no CCCD"
except Exception as e: # noqa: BLE001
notif[ch.uuid] = f"FAILED {type(e).__name__}"
out["services"].append(so)
await asyncio.sleep(10)
out["notify"] = notif
out["lost"] = lost
closing = True
await cli.bluetooth_device_disconnect(addr)
await asyncio.sleep(2)
cancel()
await cli.disconnect()
return out
async def main(host: str, peer: str, path: str) -> None:
addr = int(peer.replace(":", ""), 16)
for _attempt in range(6):
res = await once(host, addr)
if res:
break
await asyncio.sleep(10)
else:
sys.exit("FAIL: no connection")
pathlib.Path(path).write_text(json.dumps(res, indent=1), encoding="utf-8")
for s in res["services"]:
print(f"SVC {s['uuid']}")
for c in s["characteristics"]:
print(
f" CHR {c['uuid']} props=0x{c['props']:02X} read={res['reads'].get(c['uuid'], '-')}"
f" notify={res['notify'].get(c['uuid'], '-')}"
)
for d in c["descriptors"]:
print(f" DSC {d}")
asyncio.run(main(sys.argv[1], sys.argv[2], sys.argv[3]))
@@ -1,14 +0,0 @@
#!/bin/zsh
# run.sh NAME: build ../rp2_web_ble/picow.yaml as NAME, USB flash the Pico W proxy, list the GATT UUID server
# (server.yaml here) through it and compare with the ESP32 reference. Exit 0 only when the lists match.
# env: TREE, PY, PICO_SN, PEER_MAC (the server); esp32ref.json (gattlist.py through an ESP32 proxy) and apikey.txt here
D=${0:A:h}; N=$1; S=$D/..; T=${TREE:?set TREE}; PY=${PY:-$T/venv/bin/python}; C=$S/rp2_web_ble/picow.yaml
$S/pio_build.sh compile $N $C || exit 1
PORT=$($PY $S/ports.py --wait 2e8a --serial ${PICO_SN:?set PICO_SN} --timeout 30) || { echo "no pico"; exit 1; }
$S/pio_build.sh upload $N $C $PORT || exit 1
H=$($S/resolve.sh $N) || exit 1; sleep 5
rm -f $D/$N.json
API_KEY=$(cat $D/apikey.txt) timeout 600 $PY -I $D/gattlist.py $H ${PEER_MAC:?set PEER_MAC} $D/$N.json > $D/$N-list.txt
RC=$?; grep -v "^ *CHR 00002a\|DSC 00002902\|SVC 000018" $D/$N-list.txt
[ $RC = 0 ] && [ -s $D/$N.json ] || { echo "FAIL: gattlist.py exited $RC"; exit 1; }
$PY $D/cmp.py $D/esp32ref.json $D/$N.json
@@ -1,27 +0,0 @@
# GATT server (ESP32-S3) for the UUID form check: the ble_notify server (16 bit and 32-bit base UUIDs, notify) plus a
# 128 bit service whose characteristics and descriptors use 128 bit, 32-bit base (zero and nonzero low half) and
# 16 bit UUIDs, with a 2901 user description. Set the name with -s name.
packages:
notify: !include ../ble_notify/server.yaml
esp32_ble_server:
services:
- uuid: 0f1e2d3c-4b5a-6978-8796-a5b4c3d2e1f0
characteristics:
- uuid: 0f1e2d3c-4b5a-6978-8796-a5b4c3d2e1f1
read: true
value: "c128"
description: "text value"
descriptors:
- uuid: 5ec1ab01-0000-1000-8000-00805f9b34fb
value: "d32"
- uuid: 7d3e0a51-9c1b-4f2e-8a6d-1c2b3a4d5e6f
value: "d128"
- uuid: 5ec20000-0000-1000-8000-00805f9b34fb
read: true
value: "c32zero"
- uuid: 5ec2abcd-0000-1000-8000-00805f9b34fb
read: true
value: "c32"
- uuid: 2a19
read: true
value: [77]
@@ -1,79 +0,0 @@
#pragma once
#include <atomic>
#include <cstdint>
#include <cstdlib>
#include <new>
#ifdef __XTENSA__
#include "esp_cpu_utils.h"
#include "esp_debug_helpers.h"
#endif
#include "esp_timer.h"
// Counts every global operator new and, once armed, keeps the first ALLOC_CAP of them with five caller PCs
// (callers on Xtensa only; RISC-V entries keep size and time). Aligned operator new is not counted.
// See reference/recipes/heap_alloc.md.
struct AllocEntry {
uint32_t ms;
uint32_t size; // ALLOC_MARKER for a marker: pc[0] and pc[1] then hold its two values
uint32_t pc[5];
};
static constexpr uint32_t ALLOC_CAP = 1024;
static constexpr uint32_t ALLOC_MARKER = UINT32_MAX;
AllocEntry g_allocs[ALLOC_CAP];
std::atomic<uint32_t> g_alloc_n{0};
std::atomic<uint32_t> g_new_count{0};
// seq_cst with g_alloc_busy: the dump stores false then loads busy, a recorder adds busy then loads this
std::atomic<bool> g_alloc_armed{false};
uint32_t g_alloc_armed_ms = 0; // when recording started, 0 if it never did
// Recorders between claiming a slot and filling it; the dump waits for 0 after disarming.
std::atomic<uint32_t> g_alloc_busy{0};
// Claims the next slot, or returns nullptr and disarms once the buffer is full.
static inline AllocEntry *alloc_claim(uint32_t size) {
uint32_t i = g_alloc_n.fetch_add(1);
if (i >= ALLOC_CAP) {
g_alloc_armed = false;
return nullptr;
}
AllocEntry &e = g_allocs[i];
e.ms = (uint32_t) (esp_timer_get_time() / 1000);
e.size = size;
return &e;
}
void *operator new(size_t size) {
g_new_count.fetch_add(1, std::memory_order_relaxed);
g_alloc_busy.fetch_add(1); // before the armed check, so a dump that disarmed first sees it
AllocEntry *e = g_alloc_armed ? alloc_claim(size) : nullptr;
if (e != nullptr) {
e->pc[0] = e->pc[1] = e->pc[2] = e->pc[3] = e->pc[4] = 0;
#ifdef __XTENSA__
esp_backtrace_frame_t f;
esp_backtrace_get_start(&f.pc, &f.sp, &f.next_pc);
// The start frame is this function; record the five callers above it.
// Stop at the first frame that does not unwind; the slots after it stay 0.
for (int k = 0; k < 5 && f.next_pc != 0 && esp_backtrace_get_next_frame(&f); k++)
e->pc[k] = esp_cpu_process_stack_pc(f.pc);
#endif
}
g_alloc_busy.fetch_sub(1);
void *p = malloc(size ? size : 1);
if (p == nullptr)
abort();
return p;
}
void *operator new[](size_t size) { return operator new(size); }
void operator delete(void *p) noexcept { free(p); }
void operator delete[](void *p) noexcept { free(p); }
void operator delete(void *p, size_t) noexcept { free(p); }
void operator delete[](void *p, size_t) noexcept { free(p); }
// Splits the dump into phases, for example on each change of an entity's state.
static inline void alloc_marker(uint32_t a, uint32_t b) {
g_alloc_busy.fetch_add(1);
if (AllocEntry *e = g_alloc_armed ? alloc_claim(ALLOC_MARKER) : nullptr) {
e->pc[0] = a;
e->pc[1] = b;
e->pc[2] = e->pc[3] = e->pc[4] = 0;
}
g_alloc_busy.fetch_sub(1);
}
@@ -1,53 +0,0 @@
# Package for allocring.hpp: arms it ${alloc_arm_s} s after boot, reports that on a binary sensor and adds the
# dump button. See heap_alloc.md.
esphome:
includes:
- allocring.hpp
on_boot:
then:
- delay: ${alloc_arm_s}s
- lambda: |-
g_alloc_armed_ms = millis();
g_alloc_armed = true;
ESP_LOGI("allocs", "ARMED total=%u", g_new_count.load());
binary_sensor:
# Drivers wait for this before the first action, so no step runs before recording starts
- platform: template
name: Allocation recorder armed
lambda: return g_alloc_armed.load();
interval:
- interval: 30s
then:
- lambda: |-
ESP_LOGI("allocs", "total=%u kept=%u heap=%u", g_new_count.load(), (unsigned) g_alloc_n.load(),
(unsigned) heap_caps_get_free_size(MALLOC_CAP_INTERNAL));
button:
- platform: template
name: Dump allocations
on_press:
# Recording stops for good; reboot before the next run, or a second press returns the same entries.
- lambda: |-
g_alloc_armed = false;
// let recorders on other tasks finish the entry they claimed
for (int w = 0; w < 100 && g_alloc_busy.load() != 0; w++)
delay(1);
uint32_t claimed = g_alloc_n.load();
uint32_t n = std::min<uint32_t>(claimed, ALLOC_CAP);
// claimed > n means the buffer filled and recording stopped early
// armed_ms=0: the recorder never ran; busy>0: an entry may still be half written;
// buf is the buffer's address, which decode.py matches against the ELF
ESP_LOGI("allocs", "DUMP n=%u claimed=%u total=%u armed_ms=%u busy=%u buf=%08x", n, claimed,
g_new_count.load(), g_alloc_armed_ms, g_alloc_busy.load(), (unsigned) (uintptr_t) g_allocs);
for (uint32_t i = 0; i < n; i++) {
const AllocEntry &e = g_allocs[i];
ESP_LOGI("allocs", "E %u %u %u %08x %08x %08x %08x %08x", i, e.ms, e.size, e.pc[0], e.pc[1], e.pc[2],
e.pc[3], e.pc[4]);
if ((i & 15) == 15) {
delay(20); // let the log drain
App.feed_wdt(); // a full dump over a 115200 baud UART log outlasts the task watchdog
}
}
ESP_LOGI("allocs", "DUMP END");
@@ -1,108 +0,0 @@
"""Move the device's first cover to the far end and back N times with a states-only client, then save the dump.
usage: cycles_cover.py HOST CYCLES OUT env: API_KEY (unset for a plaintext API)
Waits for the recorder's armed binary sensor first. Exits 2 when a move did not reach its position (the
cover never moved, so the dump proves nothing).
"""
import asyncio
import pathlib
import sys
import time
from aioesphomeapi import (
BinarySensorInfo,
BinarySensorState,
CoverInfo,
CoverOperation,
CoverState,
)
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from apiconn import connect # noqa: E402
from heap_alloc.dump import dump # noqa: E402
# After each move, so allocations that come once the cover has stopped land in the same cycle
SETTLE_S = 8
ARM_WAIT_S = 600
async def main() -> None:
host, cycles, out = sys.argv[1], int(sys.argv[2]), sys.argv[3]
with pathlib.Path(out).open("w", encoding="utf-8") as f:
def w(s: str) -> None:
f.write(time.strftime("%H:%M:%S ") + s + "\n")
f.flush()
c = await connect(host)
ents, _ = await c.list_entities_services()
cover = next(e for e in ents if isinstance(e, CoverInfo))
armed_info = next(
e
for e in ents
if isinstance(e, BinarySensorInfo)
and e.object_id == "allocation_recorder_armed"
)
states: asyncio.Queue = asyncio.Queue()
armed = asyncio.Event()
def on_state(s) -> None:
if isinstance(s, CoverState) and s.key == cover.key:
states.put_nowait(s)
elif (
isinstance(s, BinarySensorState) and s.key == armed_info.key and s.state
):
armed.set()
c.subscribe_states(on_state)
latest = await asyncio.wait_for(states.get(), 10)
# moves made before recording starts would be missing from the dump
await asyncio.wait_for(armed.wait(), ARM_WAIT_S)
w("ARMED")
def drain() -> CoverState:
"""The newest queued state, so no stale one is read as this command's result."""
nonlocal latest
while not states.empty():
latest = states.get_nowait()
return latest
async def move(pos: float) -> bool:
drain()
w(f"CMD {pos}")
c.cover_command(cover.key, position=pos)
moved = False
deadline = time.monotonic() + 60
while (left := deadline - time.monotonic()) > 0:
try:
s = await asyncio.wait_for(states.get(), left)
except TimeoutError:
break
idle = s.current_operation == CoverOperation.IDLE
moved |= not idle
if moved and idle and abs(s.position - pos) < 0.01:
w(f"IDLE pos={s.position:.3f}")
return True
w("TIMEOUT")
return False
timeouts = 0
# start toward the far end, so the first command always moves the cover
order = (0.0, 1.0) if drain().position > 0.5 else (1.0, 0.0)
for i in range(cycles):
w(f"CYCLE {i + 1}")
for pos in order:
timeouts += not await move(pos)
await asyncio.sleep(SETTLE_S)
if timeouts:
# before the dump, which exits on its own failure
print(f"{timeouts} moves timed out", file=sys.stderr)
await dump(c, w)
await c.disconnect()
if timeouts:
sys.exit(2)
if __name__ == "__main__":
asyncio.run(main())
@@ -1,159 +0,0 @@
"""Decode an allocring dump into callers: decode.py LOG ELF [ADDR2LINE]
Prints each entry with its caller frames as function and file:line, and markers (size 0xFFFFFFFF, ALLOC_MARKER) as phase
separators.
ADDR2LINE defaults to an Xtensa one found in ESPHome's ESP-IDF tools cache, ~/.platformio or ~/.espressif.
Exits 2 when the dump is incomplete (missing header or entries, or the buffer filled during the run).
"""
from pathlib import Path
import re
import subprocess
import sys
ENTRY = re.compile(r"\bE (\d+) (\d+) (\d+) ((?:[0-9a-f]{8} ?){5})")
HEADER = re.compile(
r"\bDUMP n=(\d+) claimed=(\d+) total=\d+ armed_ms=(\d+) busy=(\d+) buf=([0-9a-f]{8})"
)
MARKER = 0xFFFFFFFF
XTENSA = 94 # ELF e_machine; the recorder walks frames only on Xtensa
def tool_roots() -> list[Path]:
roots = [Path.home() / ".platformio", Path.home() / ".espressif"]
try:
from esphome.espidf.framework import get_idf_tools_path
except ImportError:
return roots
return [get_idf_tools_path(), *roots]
def is_xtensa(elf: str) -> bool:
return int.from_bytes(Path(elf).read_bytes()[18:20], "little") == XTENSA
def find_addr2line() -> str:
for root in tool_roots():
for path in root.glob("**/bin/xtensa-esp*-elf-addr2line"):
if path.is_file():
return str(path)
sys.exit("no addr2line found; pass it as the third argument")
def short(path: str) -> str:
for marker in ("esphome/components/", "esphome/core/", "esp-idf/", "include/c++/"):
if marker in path:
return marker + path.split(marker, 1)[1]
return path
def decode(
addr2line: str, elf: str, pcs: list[str]
) -> dict[str, list[tuple[str, str]]]:
"""Every PC's frames (inlined ones too); -a prints each address before its frames."""
out = subprocess.run(
[addr2line, "-afiC", "-e", elf, *(f"0x{pc}" for pc in pcs)],
capture_output=True,
text=True,
check=True,
).stdout.splitlines()
names: dict[str, list[tuple[str, str]]] = {}
frames: list[tuple[str, str]] = []
i = 0
while i < len(out):
if out[i].startswith("0x"):
frames = names.setdefault(f"{int(out[i], 16):08x}", [])
i += 1
else:
frames.append((out[i], short(out[i + 1])))
i += 2
return names
def last_dump(text: str) -> str:
"""The newest dump in the file (a retried press or an appended file holds several); check() rejects it if cut off."""
start = max((m.start() for m in HEADER.finditer(text)), default=len(text))
return text[start:]
def check(dump: str, entries: list, xtensa: bool) -> list[str]:
"""Reasons the dump is incomplete, empty when it is whole."""
if not (header := HEADER.search(dump)):
return ["no DUMP header (log subscription started late?)"]
n, claimed, armed_ms, busy = (int(header[i]) for i in range(1, 5))
problems = []
if not armed_ms:
problems.append("the recorder was never armed (pressed before alloc_arm_s?)")
if busy:
problems.append(f"{busy} recorders were still writing an entry")
indexes = [int(e[0]) for e in entries]
if missing := sorted(set(range(n)) - set(indexes)):
problems.append(f"{len(missing)} of {n} entries missing, first {missing[:10]}")
if len(indexes) != len(set(indexes)):
problems.append("duplicate entries")
if stray := sorted(i for i in set(indexes) if i >= n):
problems.append(f"entries past n={n}: {stray[:10]}")
allocs = [pcs for _, _, size, pcs in entries if size != MARKER]
if xtensa and allocs and all(pcs[0] == "00000000" for pcs in allocs):
problems.append("no entry has a caller (frame walk failed?)")
if claimed > n:
problems.append(f"buffer filled: recording stopped after {n} entries")
if "DUMP END" not in dump:
problems.append("no DUMP END")
return problems
def main() -> None:
log, elf = sys.argv[1], sys.argv[2]
text = last_dump(Path(log).read_text(encoding="utf-8", errors="replace"))
entries = [
(idx, int(ms), int(size), pcs.split())
for idx, ms, size, pcs in ENTRY.findall(text)
]
wanted = {
pc
for _, _, size, pcs in entries
if size != MARKER
for pc in pcs
if pc != "00000000"
}
# Only Xtensa dumps carry callers; RISC-V ones still print sizes and times
names = {}
if wanted:
addr2line = sys.argv[3] if len(sys.argv) > 3 else find_addr2line()
names = decode(addr2line, elf, sorted(wanted))
for idx, ms, size, pcs in entries:
if size == MARKER:
print(
f"--- marker at {ms / 1000:.3f} s: {int(pcs[0], 16)} {int(pcs[1], 16)}"
)
continue
print(f"#{idx} {ms / 1000:.3f} s {size} B")
for pc in pcs:
if pc == "00000000":
break
for func, loc in names[pc]:
print(f" {func[:110]} {loc}")
problems = check(text, entries, is_xtensa(elf))
# addr2line names any address, so check the ELF is the build that made the dump
if wanted and (header := HEADER.search(text)):
nm = addr2line[: -len("addr2line")] + "nm"
if not Path(nm).is_file():
problems.append(f"cannot check the ELF: no {nm} next to addr2line")
elif not re.search(
rf"^0*{int(header[5], 16):x} [bBdD] g_allocs$",
subprocess.run(
[nm, elf], capture_output=True, text=True, check=True
).stdout,
re.MULTILINE,
):
problems.append(
"the ELF is not the build that made this dump (g_allocs address differs)"
)
if problems:
print("INCOMPLETE DUMP: " + "; ".join(problems), file=sys.stderr)
sys.exit(2)
if __name__ == "__main__":
main()
@@ -1,84 +0,0 @@
"""Press the "Dump allocations" button from allocring.yaml and save the dump.
usage: dump.py HOST OUT env: API_KEY (unset for a plaintext API)
"""
import asyncio
from collections.abc import Callable
import contextlib
import pathlib
import re
import sys
from aioesphomeapi import APIClient, ButtonInfo
from aioesphomeapi.model import LogLevel
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
from apiconn import connect, log_text # noqa: E402
HEADER = re.compile(r"\bDUMP n=(\d+) claimed=\d+ total=\d+ armed_ms=(\d+) busy=(\d+)")
ENTRY = re.compile(r"\bE (\d+) \d+ \d+ ")
async def dump(c: APIClient, write: Callable[[str], None], tries: int = 3) -> None:
"""Subscribe to logs, press the dump button and pass every line to WRITE until DUMP END.
subscribe_logs does not confirm the subscription and the device drops log lines when busy, so the press is
repeated (the dump is repeatable) until the header and all n entries arrived; exits 2 after TRIES presses.
"""
ents, _ = await c.list_entities_services()
button = next(
e
for e in ents
if isinstance(e, ButtonInfo) and e.object_id == "dump_allocations"
)
seen: dict = {"n": None, "idx": set()}
done = asyncio.Event()
def on_log(msg) -> None:
text = log_text(msg)
write(text)
if m := HEADER.search(text):
seen["n"], seen["idx"] = int(m[1]), set()
seen["armed"], seen["busy"] = int(m[2]) != 0, int(m[3])
elif m := ENTRY.search(text):
seen["idx"].add(int(m[1]))
elif "DUMP END" in text and seen["n"] is not None:
# without this press's header it is a late end from an earlier press
done.set()
c.subscribe_logs(on_log, log_level=LogLevel.LOG_LEVEL_INFO)
for _ in range(tries):
await asyncio.sleep(2)
seen["n"] = None
done.clear()
c.button_command(button.key)
with contextlib.suppress(TimeoutError):
await asyncio.wait_for(done.wait(), 120)
# the device drops log lines when its send buffer is full, so require every entry
if done.is_set() and seen["n"] is not None and not seen["armed"]:
print(
"the recorder was never armed (pressed before alloc_arm_s?)",
file=sys.stderr,
)
sys.exit(2)
if (
done.is_set()
and seen["n"] is not None
and not seen["busy"]
and seen["idx"] == set(range(seen["n"]))
):
return
print(f"no whole dump after {tries} presses", file=sys.stderr)
sys.exit(2)
async def main() -> None:
c = await connect(sys.argv[1])
with pathlib.Path(sys.argv[2]).open("w", encoding="utf-8") as f:
await dump(c, lambda s: f.write(s + "\n"))
await c.disconnect()
if __name__ == "__main__":
asyncio.run(main())
@@ -70,9 +70,7 @@ def frame(msg_type: int, payload: bytes) -> bytes:
return b"\x00" + varint(len(payload)) + varint(msg_type) + payload
def f_bytes(num: int, data: bytes | str) -> bytes:
if isinstance(data, str):
data = data.encode()
def f_bytes(num: int, data: bytes) -> bytes:
return varint((num << 3) | 2) + varint(len(data)) + data
@@ -1,51 +0,0 @@
"""Press the device's buttons through Home Assistant and print the script and flag states after each step.
usage: drive.py TOKENFILE NAME env: HA_URL (default http://127.0.0.1:8129)
NAME is the device name from hsw.yaml (button entity ids are button.<NAME>_long_on and so on).
"""
import pathlib
import sys
import time
sys.path.insert(
0, str(pathlib.Path(__file__).resolve().parent.parent / "mqtt" / "ha_discovery")
)
from ha_rest import req_or_exit # noqa: E402
tok = pathlib.Path(sys.argv[1]).read_text(encoding="utf-8").strip()
name = sys.argv[2].replace("-", "_") # entity ids use underscores
t0 = time.monotonic()
def state(entity: str) -> str:
return req_or_exit(f"/api/states/{entity}", token=tok)["state"]
def show(tag: str) -> None:
print(
f"{time.monotonic() - t0:6.1f}s {tag:30s} long_job={state('script.long_job'):4s}"
f" quick_job={state('script.quick_job'):4s} flag={state('input_boolean.script_ran')}",
flush=True,
)
def call(domain: str, action: str, entity: str, wait: float = 3) -> None:
req_or_exit(f"/api/services/{domain}/{action}", {"entity_id": entity}, token=tok)
time.sleep(wait)
show("start")
call("button", "press", f"button.{name}_long_on")
show("device Long on")
call("button", "press", f"button.{name}_long_off")
show("device Long off")
call("input_boolean", "turn_off", "input_boolean.script_ran", 1)
call("button", "press", f"button.{name}_long_on")
show("device Long on again")
time.sleep(20)
show("script finished on its own")
call("button", "press", f"button.{name}_quick_on")
show("device Quick on")
call("script", "turn_on", "script.long_job")
show("HA starts the script")
@@ -1,33 +0,0 @@
# Throwaway Home Assistant on port 8129: a 20 s script that sets and clears a flag, and a toggle script.
http:
server_port: 8129
api:
config:
onboarding:
frontend:
input_boolean:
script_ran:
script:
long_job:
alias: Long job
mode: single
sequence:
- action: input_boolean.turn_on
target:
entity_id: input_boolean.script_ran
- delay: 20
- action: input_boolean.turn_off
target:
entity_id: input_boolean.script_ran
quick_job:
alias: Quick job
sequence:
- action: input_boolean.toggle
target:
entity_id: input_boolean.script_ran
logger:
default: warning
logs:
homeassistant.components.esphome: debug
homeassistant.components.script: debug
homeassistant.helpers.script: info
@@ -1,36 +0,0 @@
"""Add a host build's API to a throwaway Home Assistant and allow it to call actions.
usage: haflow.py TOKENFILE env: HA_URL (default http://127.0.0.1:8129)
Runs the esphome config flow against 127.0.0.1:6053, then turns on allow_service_calls in the options flow.
"""
import pathlib
import sys
sys.path.insert(
0, str(pathlib.Path(__file__).resolve().parent.parent / "mqtt" / "ha_discovery")
)
from ha_rest import req_or_exit # noqa: E402
tok = pathlib.Path(sys.argv[1]).read_text(encoding="utf-8").strip()
flow = "/api/config/config_entries/flow"
f = req_or_exit(flow, {"handler": "esphome"}, token=tok)
r = req_or_exit(
f"{flow}/{f['flow_id']}", {"host": "127.0.0.1", "port": 6053}, token=tok
)
if r.get("type") == "form" and r.get("errors"):
sys.exit(f"config flow step {r.get('step_id')} failed: {r['errors']}")
if r.get("type") == "form": # confirmation step
r = req_or_exit(f"{flow}/{r['flow_id']}", {}, token=tok)
if r.get("type") != "create_entry":
sys.exit(
f"config flow ended with {r.get('type')} {r.get('errors') or r.get('reason')}"
)
opts = "/api/config/config_entries/options/flow"
o = req_or_exit(opts, {"handler": r["result"]["entry_id"]}, token=tok)
o = req_or_exit(f"{opts}/{o['flow_id']}", {"allow_service_calls": True}, token=tok)
if o.get("type") != "create_entry":
sys.exit(
f"options flow ended with {o.get('type')} {o.get('errors') or o.get('reason')}"
)
print("added", r.get("title"), "with allow_service_calls")
Loaded 100 of 1544 files, more files were not shown because too many files have changed in this diff. Show more